2023-07-06T01:05:31.613 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-06T01:05:31.615 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789 --name rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi --verbose --owner scheduled_rishabh@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} -- /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789/orig.config.yaml 2023-07-06T01:05:31.691 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-06T01:05:31.781 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789 branch: wip-rishabh-2023Jul5 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} email: null first_in_suite: false job_id: '7327789' kernel: client: sha1: distro flavor: default kdb: true sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul5 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul5 sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul5 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi022.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi103.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b53f236c1077f1d295baafb9e0ca9a4b447019d7 timestamp: 2023-07-05_22:21:20 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-06T01:05:31.781 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa; will attempt to use it 2023-07-06T01:05:31.782 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks 2023-07-06T01:05:31.782 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-06T01:05:31.784 INFO:teuthology.task.internal:Checking packages... 2023-07-06T01:05:31.814 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '79365bb3b803780276d69efb2cc4afe5b160de9e' 2023-07-06T01:05:31.815 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-06T01:05:31.816 INFO:teuthology.packaging:ref: None 2023-07-06T01:05:31.816 INFO:teuthology.packaging:tag: None 2023-07-06T01:05:31.816 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul5 2023-07-06T01:05:31.816 INFO:teuthology.packaging:sha1: 79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T01:05:31.817 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul5 2023-07-06T01:05:31.959 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4742.g79365bb3 2023-07-06T01:05:31.960 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-06T01:05:31.969 INFO:teuthology.task.internal:no buildpackages task found 2023-07-06T01:05:31.969 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-06T01:05:32.019 INFO:teuthology.task.internal:Saving configuration 2023-07-06T01:05:32.042 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-06T01:05:32.053 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T01:05:32.081 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 01:00:04.018713', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-06T01:05:32.108 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 01:00:04.020694', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-06T01:05:32.109 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-06T01:05:32.120 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-07-06T01:05:32.120 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-07-06T01:05:32.120 INFO:teuthology.run_tasks:Running task console_log... 2023-07-06T01:05:32.241 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa88d7e5280>, signals=[15]) 2023-07-06T01:05:32.243 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-06T01:05:32.251 INFO:teuthology.task.internal:Opening connections... 2023-07-06T01:05:32.252 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:05:32.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T01:05:32.324 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2023-07-06T01:05:32.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T01:05:32.392 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-06T01:05:32.402 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2023-07-06T01:05:32.425 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2023-07-06T01:05:32.426 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2023-07-06T01:05:32.485 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="Red Hat Enterprise Linux" 2023-07-06T01:05:32.486 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="8.6 (Ootpa)" 2023-07-06T01:05:32.486 INFO:teuthology.orchestra.run.smithi022.stdout:ID="rhel" 2023-07-06T01:05:32.486 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="fedora" 2023-07-06T01:05:32.486 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="8.6" 2023-07-06T01:05:32.486 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el8" 2023-07-06T01:05:32.486 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-06T01:05:32.486 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2023-07-06T01:05:32.487 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-06T01:05:32.487 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://www.redhat.com/" 2023-07-06T01:05:32.487 INFO:teuthology.orchestra.run.smithi022.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-06T01:05:32.487 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-06T01:05:32.487 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:05:32.487 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-06T01:05:32.487 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-06T01:05:32.488 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-06T01:05:32.488 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-06T01:05:32.488 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2023-07-06T01:05:32.512 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2023-07-06T01:05:32.533 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2023-07-06T01:05:32.533 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2023-07-06T01:05:32.592 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="Red Hat Enterprise Linux" 2023-07-06T01:05:32.592 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="8.6 (Ootpa)" 2023-07-06T01:05:32.592 INFO:teuthology.orchestra.run.smithi103.stdout:ID="rhel" 2023-07-06T01:05:32.592 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="fedora" 2023-07-06T01:05:32.592 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="8.6" 2023-07-06T01:05:32.593 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el8" 2023-07-06T01:05:32.593 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-06T01:05:32.593 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2023-07-06T01:05:32.593 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-06T01:05:32.593 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://www.redhat.com/" 2023-07-06T01:05:32.593 INFO:teuthology.orchestra.run.smithi103.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-06T01:05:32.593 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-06T01:05:32.594 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:05:32.594 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-06T01:05:32.594 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-06T01:05:32.594 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-06T01:05:32.594 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-06T01:05:32.594 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2023-07-06T01:05:32.620 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-06T01:05:32.636 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-06T01:05:32.669 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-06T01:05:32.669 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2023-07-06T01:05:32.671 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2023-07-06T01:05:32.690 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-06T01:05:32.710 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-06T01:05:32.711 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2023-07-06T01:05:32.732 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2023-07-06T01:05:32.764 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-06T01:05:32.829 INFO:teuthology.run_tasks:Running task kernel... 2023-07-06T01:05:32.849 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'} 2023-07-06T01:05:32.850 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-07-06T01:05:32.850 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-07-06T01:05:32.851 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-06T01:05:32.851 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-06T01:05:32.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:05:32.871 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2023-07-06T01:05:32.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:05:32.872 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2023-07-06T01:05:32.930 INFO:teuthology.orchestra.run.smithi103.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-06T01:05:32.930 INFO:teuthology.task.kernel:Running kernel on smithi103: 4.18.0-372.9.1.el8.x86_64 2023-07-06T01:05:32.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum install -y kernel 2023-07-06T01:05:32.931 INFO:teuthology.orchestra.run.smithi022.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-06T01:05:32.932 INFO:teuthology.task.kernel:Running kernel on smithi022: 4.18.0-372.9.1.el8.x86_64 2023-07-06T01:05:32.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum install -y kernel 2023-07-06T01:05:34.077 INFO:teuthology.orchestra.run.smithi103.stdout:Updating Subscription Management repositories. 2023-07-06T01:05:34.077 INFO:teuthology.orchestra.run.smithi103.stdout:Unable to read consumer identity 2023-07-06T01:05:34.151 INFO:teuthology.orchestra.run.smithi022.stdout:Updating Subscription Management repositories. 2023-07-06T01:05:34.151 INFO:teuthology.orchestra.run.smithi022.stdout:Unable to read consumer identity 2023-07-06T01:05:37.875 INFO:teuthology.orchestra.run.smithi103.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-07-06T01:05:38.047 INFO:teuthology.orchestra.run.smithi022.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T01:05:39.984 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 16 MB 00:02 2023-07-06T01:05:40.172 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 7.4 MB/s | 16 MB 00:02 2023-07-06T01:05:43.459 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 420 kB/s | 24 kB 00:00 2023-07-06T01:05:43.830 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 435 kB/s | 24 kB 00:00 2023-07-06T01:05:44.111 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-07-06T01:05:44.170 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2023-07-06T01:05:44.171 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2023-07-06T01:05:44.172 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2023-07-06T01:05:44.292 DEBUG:teuthology.orchestra.run.smithi103:> echo no | sudo yum reinstall kernel || true 2023-07-06T01:05:44.541 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-07-06T01:05:44.585 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2023-07-06T01:05:44.596 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2023-07-06T01:05:44.597 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2023-07-06T01:05:44.662 INFO:teuthology.orchestra.run.smithi103.stdout:Updating Subscription Management repositories. 2023-07-06T01:05:44.662 INFO:teuthology.orchestra.run.smithi103.stdout:Unable to read consumer identity 2023-07-06T01:05:44.712 DEBUG:teuthology.orchestra.run.smithi022:> echo no | sudo yum reinstall kernel || true 2023-07-06T01:05:44.921 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:01 ago on Thu 06 Jul 2023 01:05:43 AM UTC. 2023-07-06T01:05:44.962 INFO:teuthology.orchestra.run.smithi103.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-06T01:05:45.006 INFO:teuthology.orchestra.run.smithi103.stderr:Error: No packages marked for reinstall. 2023-07-06T01:05:45.051 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum reinstall -y kernel || true 2023-07-06T01:05:45.062 INFO:teuthology.orchestra.run.smithi022.stdout:Updating Subscription Management repositories. 2023-07-06T01:05:45.062 INFO:teuthology.orchestra.run.smithi022.stdout:Unable to read consumer identity 2023-07-06T01:05:45.344 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:02 ago on Thu 06 Jul 2023 01:05:43 AM UTC. 2023-07-06T01:05:45.379 INFO:teuthology.orchestra.run.smithi103.stdout:Updating Subscription Management repositories. 2023-07-06T01:05:45.379 INFO:teuthology.orchestra.run.smithi103.stdout:Unable to read consumer identity 2023-07-06T01:05:45.389 INFO:teuthology.orchestra.run.smithi022.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-06T01:05:45.419 INFO:teuthology.orchestra.run.smithi022.stderr:Error: No packages marked for reinstall. 2023-07-06T01:05:45.466 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum reinstall -y kernel || true 2023-07-06T01:05:45.635 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:02 ago on Thu 06 Jul 2023 01:05:43 AM UTC. 2023-07-06T01:05:45.674 INFO:teuthology.orchestra.run.smithi103.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-06T01:05:45.700 INFO:teuthology.orchestra.run.smithi103.stderr:Error: No packages marked for reinstall. 2023-07-06T01:05:45.745 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2023-07-06T01:05:45.815 INFO:teuthology.orchestra.run.smithi022.stdout:Updating Subscription Management repositories. 2023-07-06T01:05:45.816 INFO:teuthology.orchestra.run.smithi022.stdout:Unable to read consumer identity 2023-07-06T01:05:45.825 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-07-06T01:05:45.826 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-07-06T01:05:45.826 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-07-06T01:05:45.826 DEBUG:teuthology.parallel:result is None 2023-07-06T01:05:46.088 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:03 ago on Thu 06 Jul 2023 01:05:43 AM UTC. 2023-07-06T01:05:46.130 INFO:teuthology.orchestra.run.smithi022.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-06T01:05:46.149 INFO:teuthology.orchestra.run.smithi022.stderr:Error: No packages marked for reinstall. 2023-07-06T01:05:46.202 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2023-07-06T01:05:46.285 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-07-06T01:05:46.285 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-07-06T01:05:46.286 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-07-06T01:05:46.286 DEBUG:teuthology.parallel:result is None 2023-07-06T01:05:46.286 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-06T01:05:46.294 INFO:teuthology.task.internal:Creating test directory... 2023-07-06T01:05:46.295 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-06T01:05:46.297 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-06T01:05:46.313 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-06T01:05:46.322 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-06T01:05:46.338 INFO:teuthology.task.internal:Creating archive directory... 2023-07-06T01:05:46.339 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-06T01:05:46.354 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-06T01:05:46.377 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-06T01:05:46.385 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-06T01:05:46.385 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-06T01:05:46.410 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-06T01:05:46.446 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T01:05:46.451 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T01:05:46.464 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T01:05:46.468 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T01:05:46.469 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-06T01:05:46.477 INFO:teuthology.task.internal:Configuring sudo... 2023-07-06T01:05:46.477 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-06T01:05:46.507 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-06T01:05:46.536 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-06T01:05:46.546 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-06T01:05:46.546 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-06T01:05:46.574 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-06T01:05:46.593 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T01:05:46.671 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T01:05:46.722 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T01:05:46.809 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T01:05:46.848 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:05:46.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-06T01:05:46.917 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T01:05:46.966 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T01:05:47.011 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T01:05:47.097 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T01:05:47.134 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:05:47.135 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-06T01:05:47.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2023-07-06T01:05:47.204 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2023-07-06T01:05:47.260 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T01:05:47.302 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T01:05:47.660 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-06T01:05:47.671 INFO:teuthology.task.internal:Starting timer... 2023-07-06T01:05:47.671 INFO:teuthology.run_tasks:Running task pcp... 2023-07-06T01:05:47.700 INFO:teuthology.run_tasks:Running task selinux... 2023-07-06T01:05:47.737 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-06T01:05:47.738 DEBUG:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2023-07-06T01:05:47.804 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: 2023-07-06T01:05:47.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2023-07-06T01:05:47.862 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2023-07-06T01:05:47.863 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-06T01:05:47.864 DEBUG:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2023-07-06T01:05:47.885 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2023-07-06T01:05:47.885 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2023-07-06T01:05:47.910 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2023-07-06T01:05:47.910 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi022.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi103.front.sepia.ceph.com': 'permissive'} 2023-07-06T01:05:47.911 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T01:05:47.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:05:47.953 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T01:05:47.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:05:47.981 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-06T01:05:47.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2023-07-06T01:05:48.021 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2023-07-06T01:05:48.049 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-06T01:05:48.061 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-07-06T01:05:48.062 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-06T01:05:48.082 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-06T01:05:48.084 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi103.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-06T01:16:12.283 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi103.front.sepia.ceph.com')] 2023-07-06T01:16:12.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-07-06T01:16:12.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T01:16:12.354 DEBUG:teuthology.orchestra.run.smithi022:> true 2023-07-06T01:16:12.427 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2023-07-06T01:16:12.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2023-07-06T01:16:12.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T01:16:12.496 DEBUG:teuthology.orchestra.run.smithi103:> true 2023-07-06T01:16:12.562 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2023-07-06T01:16:12.563 INFO:teuthology.run_tasks:Running task clock... 2023-07-06T01:16:12.575 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-06T01:16:12.575 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-06T01:16:12.576 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T01:16:12.579 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-06T01:16:12.579 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T01:16:12.614 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-06T01:16:12.631 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-06T01:16:12.634 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-06T01:16:12.649 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-06T01:16:12.698 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: ntpd: command not found 2023-07-06T01:16:12.711 INFO:teuthology.orchestra.run.smithi022.stdout:506 Cannot talk to daemon 2023-07-06T01:16:12.727 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-06T01:16:12.727 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2023-07-06T01:16:12.738 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2023-07-06T01:16:12.742 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-06T01:16:12.753 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-06T01:16:12.767 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-06T01:16:12.809 INFO:teuthology.orchestra.run.smithi022.stderr:bash: ntpq: command not found 2023-07-06T01:16:12.813 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T01:16:12.813 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2023-07-06T01:16:12.813 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T01:16:12.813 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T01:16:12.813 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T01:16:12.813 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T01:16:12.851 INFO:teuthology.orchestra.run.smithi103.stderr:bash: ntpq: command not found 2023-07-06T01:16:12.853 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T01:16:12.853 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2023-07-06T01:16:12.854 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T01:16:12.854 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T01:16:12.854 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T01:16:12.854 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T01:16:12.855 INFO:teuthology.run_tasks:Running task install... 2023-07-06T01:16:12.864 DEBUG:teuthology.task.install:project ceph 2023-07-06T01:16:12.865 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}} 2023-07-06T01:16:12.865 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'} 2023-07-06T01:16:12.865 INFO:teuthology.task.install:Using flavor: default 2023-07-06T01:16:12.875 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-06T01:16:12.876 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-06T01:16:12.876 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'tag': None, 'wait_for_package': False} 2023-07-06T01:16:12.877 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T01:16:12.879 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'tag': None, 'wait_for_package': False} 2023-07-06T01:16:12.879 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T01:16:13.024 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/ 2023-07-06T01:16:13.024 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4742.g79365bb3 2023-07-06T01:16:13.035 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/ 2023-07-06T01:16:13.036 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4742.g79365bb3 2023-07-06T01:16:13.148 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-06T01:16:13.149 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:16:13.149 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-06T01:16:13.179 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-06T01:16:13.179 DEBUG:teuthology.orchestra.run.smithi022:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/79365bb3b803780276d69efb2cc4afe5b160de9e/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-06T01:16:13.198 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul5/79365bb3b803780276d69efb2cc4afe5b160de9e/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-06T01:16:13.199 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:16:13.199 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-06T01:16:13.227 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-06T01:16:13.227 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/79365bb3b803780276d69efb2cc4afe5b160de9e/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-06T01:16:13.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-06T01:16:13.294 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-06T01:16:13.328 DEBUG:teuthology.orchestra.run.smithi022:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-06T01:16:13.370 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-06T01:16:13.395 INFO:teuthology.orchestra.run.smithi022.stdout:check_obsoletes = 1 2023-07-06T01:16:13.396 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum clean all 2023-07-06T01:16:13.436 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2023-07-06T01:16:13.437 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2023-07-06T01:16:13.788 INFO:teuthology.orchestra.run.smithi022.stdout:Updating Subscription Management repositories. 2023-07-06T01:16:13.811 INFO:teuthology.orchestra.run.smithi103.stdout:Updating Subscription Management repositories. 2023-07-06T01:16:14.104 INFO:teuthology.orchestra.run.smithi103.stdout:44 files removed 2023-07-06T01:16:14.107 INFO:teuthology.orchestra.run.smithi022.stdout:44 files removed 2023-07-06T01:16:14.150 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-06T01:16:14.155 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-06T01:16:14.480 INFO:teuthology.orchestra.run.smithi103.stdout:Updating Subscription Management repositories. 2023-07-06T01:16:14.505 INFO:teuthology.orchestra.run.smithi022.stdout:Updating Subscription Management repositories. 2023-07-06T01:16:15.279 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 231 kB/s | 81 kB 00:00 2023-07-06T01:16:15.359 INFO:teuthology.orchestra.run.smithi022.stdout:ceph packages for x86_64 191 kB/s | 81 kB 00:00 2023-07-06T01:16:15.638 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 29 kB/s | 9.4 kB 00:00 2023-07-06T01:16:15.651 INFO:teuthology.orchestra.run.smithi022.stdout:ceph noarch packages 36 kB/s | 9.4 kB 00:00 2023-07-06T01:16:15.902 INFO:teuthology.orchestra.run.smithi022.stdout:ceph source packages 6.6 kB/s | 1.5 kB 00:00 2023-07-06T01:16:15.939 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 5.3 kB/s | 1.5 kB 00:00 2023-07-06T01:16:18.303 INFO:teuthology.orchestra.run.smithi022.stdout:Copr repo for python3-asyncssh owned by ceph 1.5 kB/s | 3.5 kB 00:02 2023-07-06T01:16:18.547 INFO:teuthology.orchestra.run.smithi103.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T01:16:19.554 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-07-06T01:16:19.840 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-07-06T01:16:23.154 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 387 kB/s | 24 kB 00:00 2023-07-06T01:16:23.257 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 376 kB/s | 24 kB 00:00 2023-07-06T01:16:24.064 INFO:teuthology.orchestra.run.smithi022.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 57 MB/s | 47 MB 00:00 2023-07-06T01:16:24.176 INFO:teuthology.orchestra.run.smithi103.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-07-06T01:16:32.360 INFO:teuthology.orchestra.run.smithi103.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 MB/s | 47 MB 00:00 2023-07-06T01:16:32.540 INFO:teuthology.orchestra.run.smithi022.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 MB/s | 53 MB 00:00 2023-07-06T01:16:43.299 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T01:16:43.300 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T01:16:43.314 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-06T01:16:43.316 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-06T01:16:43.317 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-06T01:16:43.318 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T01:16:43.320 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T01:16:43.323 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T01:16:43.324 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T01:16:43.459 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2023-07-06T01:16:43.471 INFO:teuthology.orchestra.run.smithi103.stdout:=========================================================================================================================== 2023-07-06T01:16:43.471 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2023-07-06T01:16:43.471 INFO:teuthology.orchestra.run.smithi103.stdout:=========================================================================================================================== 2023-07-06T01:16:43.471 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2023-07-06T01:16:43.472 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-06T01:16:43.472 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-06T01:16:43.472 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.4 k 2023-07-06T01:16:43.472 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 5.4 M 2023-07-06T01:16:43.472 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 847 k 2023-07-06T01:16:43.472 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 165 k 2023-07-06T01:16:43.472 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 1.5 M 2023-07-06T01:16:43.473 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 129 k 2023-07-06T01:16:43.473 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 1.7 M 2023-07-06T01:16:43.473 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 7.4 M 2023-07-06T01:16:43.473 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 47 k 2023-07-06T01:16:43.473 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 13 M 2023-07-06T01:16:43.473 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 48 M 2023-07-06T01:16:43.473 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 258 k 2023-07-06T01:16:43.474 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 166 k 2023-07-06T01:16:43.474 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 258 k 2023-07-06T01:16:43.474 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 18 k 2023-07-06T01:16:43.474 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-06T01:16:43.474 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-06T01:16:43.474 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-06T01:16:43.474 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 31 k 2023-07-06T01:16:43.474 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 770 k 2023-07-06T01:16:43.475 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 139 k 2023-07-06T01:16:43.475 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-06T01:16:43.475 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-06T01:16:43.475 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-06T01:16:43.475 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-06T01:16:43.475 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-06T01:16:43.475 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-06T01:16:43.475 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 197 k 2023-07-06T01:16:43.476 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 370 k 2023-07-06T01:16:43.476 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 364 k 2023-07-06T01:16:43.476 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 107 k 2023-07-06T01:16:43.476 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 94 k 2023-07-06T01:16:43.476 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.5 M 2023-07-06T01:16:43.476 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 179 k 2023-07-06T01:16:43.476 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-06T01:16:43.477 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2023-07-06T01:16:43.477 INFO:teuthology.orchestra.run.smithi103.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-06T01:16:43.477 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.7 M 2023-07-06T01:16:43.477 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.6 M 2023-07-06T01:16:43.477 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2023-07-06T01:16:43.477 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-06T01:16:43.477 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 22 M 2023-07-06T01:16:43.478 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 23 k 2023-07-06T01:16:43.478 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 2.3 M 2023-07-06T01:16:43.478 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 240 k 2023-07-06T01:16:43.478 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 4.6 M 2023-07-06T01:16:43.478 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 18 M 2023-07-06T01:16:43.478 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 15 k 2023-07-06T01:16:43.478 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 24 k 2023-07-06T01:16:43.478 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-06T01:16:43.479 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-06T01:16:43.479 INFO:teuthology.orchestra.run.smithi103.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-06T01:16:43.479 INFO:teuthology.orchestra.run.smithi103.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-06T01:16:43.479 INFO:teuthology.orchestra.run.smithi103.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-06T01:16:43.479 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 137 k 2023-07-06T01:16:43.479 INFO:teuthology.orchestra.run.smithi103.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-06T01:16:43.479 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-06T01:16:43.479 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-06T01:16:43.480 INFO:teuthology.orchestra.run.smithi103.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-06T01:16:43.480 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-06T01:16:43.480 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-06T01:16:43.480 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 475 k 2023-07-06T01:16:43.480 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-06T01:16:43.480 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.7 M 2023-07-06T01:16:43.480 INFO:teuthology.orchestra.run.smithi103.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-06T01:16:43.481 INFO:teuthology.orchestra.run.smithi103.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-06T01:16:43.481 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-06T01:16:43.481 INFO:teuthology.orchestra.run.smithi103.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-06T01:16:43.481 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-06T01:16:43.481 INFO:teuthology.orchestra.run.smithi103.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-06T01:16:43.481 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-06T01:16:43.481 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-06T01:16:43.481 INFO:teuthology.orchestra.run.smithi103.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-06T01:16:43.482 INFO:teuthology.orchestra.run.smithi103.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-06T01:16:43.482 INFO:teuthology.orchestra.run.smithi103.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-06T01:16:43.482 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-06T01:16:43.482 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-06T01:16:43.482 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-06T01:16:43.482 INFO:teuthology.orchestra.run.smithi103.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-06T01:16:43.482 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-06T01:16:43.482 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 45 k 2023-07-06T01:16:43.483 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 119 k 2023-07-06T01:16:43.483 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-06T01:16:43.483 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-06T01:16:43.483 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-06T01:16:43.483 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-06T01:16:43.483 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-06T01:16:43.483 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-06T01:16:43.484 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-06T01:16:43.484 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-06T01:16:43.484 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-06T01:16:43.484 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-06T01:16:43.484 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-06T01:16:43.484 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T01:16:43.484 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-06T01:16:43.484 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-06T01:16:43.485 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-06T01:16:43.485 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-06T01:16:43.485 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-06T01:16:43.485 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-06T01:16:43.485 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-06T01:16:43.485 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-06T01:16:43.485 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-06T01:16:43.485 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-06T01:16:43.486 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-06T01:16:43.486 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-06T01:16:43.486 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-06T01:16:43.486 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-06T01:16:43.486 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-06T01:16:43.486 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-06T01:16:43.486 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-06T01:16:43.487 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-06T01:16:43.487 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T01:16:43.487 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-06T01:16:43.487 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-06T01:16:43.487 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-06T01:16:43.487 INFO:teuthology.orchestra.run.smithi103.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-06T01:16:43.487 INFO:teuthology.orchestra.run.smithi103.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-06T01:16:43.487 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-06T01:16:43.488 INFO:teuthology.orchestra.run.smithi103.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-06T01:16:43.488 INFO:teuthology.orchestra.run.smithi103.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-06T01:16:43.488 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-06T01:16:43.488 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-06T01:16:43.488 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-06T01:16:43.488 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-06T01:16:43.488 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-06T01:16:43.488 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-06T01:16:43.489 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-06T01:16:43.489 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-06T01:16:43.489 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-06T01:16:43.489 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-06T01:16:43.489 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling module streams: 2023-07-06T01:16:43.489 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql 10 2023-07-06T01:16:43.489 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:16:43.490 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2023-07-06T01:16:43.490 INFO:teuthology.orchestra.run.smithi103.stdout:=========================================================================================================================== 2023-07-06T01:16:43.490 INFO:teuthology.orchestra.run.smithi103.stdout:Install 128 Packages 2023-07-06T01:16:43.490 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 3 Packages 2023-07-06T01:16:43.490 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:16:43.491 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 205 M 2023-07-06T01:16:43.491 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2023-07-06T01:16:44.010 INFO:teuthology.orchestra.run.smithi103.stdout:(1/131): ceph-18.0.0-4742.g79365bb3.el8.x86_64. 34 kB/s | 6.4 kB 00:00 2023-07-06T01:16:44.313 INFO:teuthology.orchestra.run.smithi022.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T01:16:44.314 INFO:teuthology.orchestra.run.smithi022.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T01:16:44.319 INFO:teuthology.orchestra.run.smithi103.stdout:(2/131): ceph-fuse-18.0.0-4742.g79365bb3.el8.x8 2.7 MB/s | 847 kB 00:00 2023-07-06T01:16:44.329 INFO:teuthology.orchestra.run.smithi022.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-06T01:16:44.330 INFO:teuthology.orchestra.run.smithi022.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-06T01:16:44.331 INFO:teuthology.orchestra.run.smithi022.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-06T01:16:44.333 INFO:teuthology.orchestra.run.smithi022.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T01:16:44.335 INFO:teuthology.orchestra.run.smithi022.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T01:16:44.338 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T01:16:44.339 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T01:16:44.411 INFO:teuthology.orchestra.run.smithi103.stdout:(3/131): ceph-base-18.0.0-4742.g79365bb3.el8.x8 9.2 MB/s | 5.4 MB 00:00 2023-07-06T01:16:44.436 INFO:teuthology.orchestra.run.smithi103.stdout:(4/131): ceph-immutable-object-cache-18.0.0-474 1.4 MB/s | 165 kB 00:00 2023-07-06T01:16:44.484 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2023-07-06T01:16:44.497 INFO:teuthology.orchestra.run.smithi022.stdout:=========================================================================================================================== 2023-07-06T01:16:44.497 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2023-07-06T01:16:44.497 INFO:teuthology.orchestra.run.smithi022.stdout:=========================================================================================================================== 2023-07-06T01:16:44.498 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2023-07-06T01:16:44.498 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-06T01:16:44.498 INFO:teuthology.orchestra.run.smithi022.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-06T01:16:44.498 INFO:teuthology.orchestra.run.smithi022.stdout: ceph x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.4 k 2023-07-06T01:16:44.498 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 5.4 M 2023-07-06T01:16:44.498 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 847 k 2023-07-06T01:16:44.498 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 165 k 2023-07-06T01:16:44.499 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 1.5 M 2023-07-06T01:16:44.499 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 129 k 2023-07-06T01:16:44.499 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 1.7 M 2023-07-06T01:16:44.499 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 7.4 M 2023-07-06T01:16:44.499 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 47 k 2023-07-06T01:16:44.499 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 13 M 2023-07-06T01:16:44.499 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 48 M 2023-07-06T01:16:44.500 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 258 k 2023-07-06T01:16:44.500 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 166 k 2023-07-06T01:16:44.500 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 258 k 2023-07-06T01:16:44.500 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 18 k 2023-07-06T01:16:44.500 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-06T01:16:44.500 INFO:teuthology.orchestra.run.smithi022.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-06T01:16:44.500 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-06T01:16:44.501 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 31 k 2023-07-06T01:16:44.501 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 770 k 2023-07-06T01:16:44.501 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 139 k 2023-07-06T01:16:44.501 INFO:teuthology.orchestra.run.smithi022.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-06T01:16:44.501 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-06T01:16:44.501 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-06T01:16:44.501 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-06T01:16:44.501 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-06T01:16:44.502 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-06T01:16:44.502 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 197 k 2023-07-06T01:16:44.502 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 370 k 2023-07-06T01:16:44.502 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 364 k 2023-07-06T01:16:44.502 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 107 k 2023-07-06T01:16:44.502 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 94 k 2023-07-06T01:16:44.502 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.5 M 2023-07-06T01:16:44.503 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 179 k 2023-07-06T01:16:44.503 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-06T01:16:44.503 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrading: 2023-07-06T01:16:44.503 INFO:teuthology.orchestra.run.smithi022.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-06T01:16:44.503 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.7 M 2023-07-06T01:16:44.503 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 3.6 M 2023-07-06T01:16:44.503 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2023-07-06T01:16:44.503 INFO:teuthology.orchestra.run.smithi022.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-06T01:16:44.504 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 22 M 2023-07-06T01:16:44.504 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 23 k 2023-07-06T01:16:44.504 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 2.3 M 2023-07-06T01:16:44.504 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 240 k 2023-07-06T01:16:44.504 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 4.6 M 2023-07-06T01:16:44.504 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 18 M 2023-07-06T01:16:44.504 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4742.g79365bb3.el8 ceph-noarch 15 k 2023-07-06T01:16:44.505 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 24 k 2023-07-06T01:16:44.505 INFO:teuthology.orchestra.run.smithi022.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-06T01:16:44.505 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-06T01:16:44.505 INFO:teuthology.orchestra.run.smithi022.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-06T01:16:44.505 INFO:teuthology.orchestra.run.smithi022.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-06T01:16:44.505 INFO:teuthology.orchestra.run.smithi022.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-06T01:16:44.505 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 137 k 2023-07-06T01:16:44.506 INFO:teuthology.orchestra.run.smithi022.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-06T01:16:44.506 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-06T01:16:44.506 INFO:teuthology.orchestra.run.smithi022.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-06T01:16:44.506 INFO:teuthology.orchestra.run.smithi022.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-06T01:16:44.506 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-06T01:16:44.506 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-06T01:16:44.506 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 475 k 2023-07-06T01:16:44.506 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-06T01:16:44.507 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2 x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 6.7 M 2023-07-06T01:16:44.507 INFO:teuthology.orchestra.run.smithi022.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-06T01:16:44.507 INFO:teuthology.orchestra.run.smithi022.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-06T01:16:44.507 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-06T01:16:44.507 INFO:teuthology.orchestra.run.smithi022.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-06T01:16:44.507 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-06T01:16:44.507 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-06T01:16:44.507 INFO:teuthology.orchestra.run.smithi022.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-06T01:16:44.508 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-06T01:16:44.508 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-06T01:16:44.508 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-06T01:16:44.508 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-06T01:16:44.508 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-06T01:16:44.508 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-06T01:16:44.508 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-06T01:16:44.509 INFO:teuthology.orchestra.run.smithi022.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-06T01:16:44.509 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-06T01:16:44.509 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 45 k 2023-07-06T01:16:44.509 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common x86_64 2:18.0.0-4742.g79365bb3.el8 ceph 119 k 2023-07-06T01:16:44.509 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-06T01:16:44.509 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-06T01:16:44.509 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-06T01:16:44.510 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-06T01:16:44.510 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-06T01:16:44.510 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-06T01:16:44.510 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-06T01:16:44.510 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-06T01:16:44.510 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-06T01:16:44.510 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-06T01:16:44.510 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-06T01:16:44.511 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T01:16:44.511 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-06T01:16:44.511 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-06T01:16:44.511 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-06T01:16:44.511 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-06T01:16:44.511 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-06T01:16:44.511 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-06T01:16:44.511 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-06T01:16:44.512 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-06T01:16:44.512 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-06T01:16:44.512 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-06T01:16:44.512 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-06T01:16:44.512 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-06T01:16:44.512 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-06T01:16:44.512 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-06T01:16:44.513 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-06T01:16:44.513 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-06T01:16:44.513 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-06T01:16:44.513 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-06T01:16:44.513 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T01:16:44.513 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-06T01:16:44.513 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-06T01:16:44.513 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-06T01:16:44.514 INFO:teuthology.orchestra.run.smithi022.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-06T01:16:44.514 INFO:teuthology.orchestra.run.smithi022.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-06T01:16:44.514 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-06T01:16:44.514 INFO:teuthology.orchestra.run.smithi022.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-06T01:16:44.514 INFO:teuthology.orchestra.run.smithi022.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-06T01:16:44.514 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-06T01:16:44.514 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-06T01:16:44.514 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-06T01:16:44.515 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-06T01:16:44.515 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-06T01:16:44.515 INFO:teuthology.orchestra.run.smithi022.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-06T01:16:44.515 INFO:teuthology.orchestra.run.smithi022.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-06T01:16:44.515 INFO:teuthology.orchestra.run.smithi022.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-06T01:16:44.515 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-06T01:16:44.515 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-06T01:16:44.516 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling module streams: 2023-07-06T01:16:44.516 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql 10 2023-07-06T01:16:44.516 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:16:44.516 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2023-07-06T01:16:44.516 INFO:teuthology.orchestra.run.smithi022.stdout:=========================================================================================================================== 2023-07-06T01:16:44.516 INFO:teuthology.orchestra.run.smithi022.stdout:Install 128 Packages 2023-07-06T01:16:44.516 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade 3 Packages 2023-07-06T01:16:44.517 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:16:44.517 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 205 M 2023-07-06T01:16:44.517 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2023-07-06T01:16:44.628 INFO:teuthology.orchestra.run.smithi103.stdout:(5/131): ceph-mds-18.0.0-4742.g79365bb3.el8.x86 11 MB/s | 2.3 MB 00:00 2023-07-06T01:16:44.703 INFO:teuthology.orchestra.run.smithi103.stdout:(6/131): ceph-mgr-18.0.0-4742.g79365bb3.el8.x86 5.8 MB/s | 1.5 MB 00:00 2023-07-06T01:16:44.731 INFO:teuthology.orchestra.run.smithi022.stdout:(1/131): ceph-18.0.0-4742.g79365bb3.el8.x86_64. 34 kB/s | 6.4 kB 00:00 2023-07-06T01:16:45.079 INFO:teuthology.orchestra.run.smithi103.stdout:(7/131): ceph-mon-18.0.0-4742.g79365bb3.el8.x86 10 MB/s | 4.6 MB 00:00 2023-07-06T01:16:45.590 INFO:teuthology.orchestra.run.smithi022.stdout:(2/131): ceph-fuse-18.0.0-4742.g79365bb3.el8.x8 988 kB/s | 847 kB 00:00 2023-07-06T01:16:45.798 INFO:teuthology.orchestra.run.smithi022.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 794 kB/s | 165 kB 00:00 2023-07-06T01:16:46.471 INFO:teuthology.orchestra.run.smithi103.stdout:(8/131): ceph-radosgw-18.0.0-4742.g79365bb3.el8 9.7 MB/s | 13 MB 00:01 2023-07-06T01:16:46.529 INFO:teuthology.orchestra.run.smithi103.stdout:(9/131): ceph-selinux-18.0.0-4742.g79365bb3.el8 418 kB/s | 24 kB 00:00 2023-07-06T01:16:46.814 INFO:teuthology.orchestra.run.smithi103.stdout:(10/131): ceph-common-18.0.0-4742.g79365bb3.el8 7.4 MB/s | 22 MB 00:02 2023-07-06T01:16:46.922 INFO:teuthology.orchestra.run.smithi103.stdout:(11/131): cephfs-mirror-18.0.0-4742.g79365bb3.e 2.3 MB/s | 258 kB 00:00 2023-07-06T01:16:47.005 INFO:teuthology.orchestra.run.smithi103.stdout:(12/131): libcephfs-devel-18.0.0-4742.g79365bb3 372 kB/s | 31 kB 00:00 2023-07-06T01:16:47.090 INFO:teuthology.orchestra.run.smithi022.stdout:(4/131): ceph-base-18.0.0-4742.g79365bb3.el8.x8 2.1 MB/s | 5.4 MB 00:02 2023-07-06T01:16:47.214 INFO:teuthology.orchestra.run.smithi103.stdout:(13/131): libcephfs2-18.0.0-4742.g79365bb3.el8. 3.6 MB/s | 770 kB 00:00 2023-07-06T01:16:47.314 INFO:teuthology.orchestra.run.smithi103.stdout:(14/131): libcephsqlite-18.0.0-4742.g79365bb3.e 1.3 MB/s | 137 kB 00:00 2023-07-06T01:16:47.423 INFO:teuthology.orchestra.run.smithi103.stdout:(15/131): librados-devel-18.0.0-4742.g79365bb3. 1.3 MB/s | 139 kB 00:00 2023-07-06T01:16:47.474 INFO:teuthology.orchestra.run.smithi022.stdout:(5/131): ceph-mds-18.0.0-4742.g79365bb3.el8.x86 1.4 MB/s | 2.3 MB 00:01 2023-07-06T01:16:47.674 INFO:teuthology.orchestra.run.smithi103.stdout:(16/131): ceph-osd-18.0.0-4742.g79365bb3.el8.x8 6.0 MB/s | 18 MB 00:02 2023-07-06T01:16:47.691 INFO:teuthology.orchestra.run.smithi022.stdout:(6/131): ceph-mgr-18.0.0-4742.g79365bb3.el8.x86 2.6 MB/s | 1.5 MB 00:00 2023-07-06T01:16:47.707 INFO:teuthology.orchestra.run.smithi103.stdout:(17/131): libradosstriper1-18.0.0-4742.g79365bb 1.6 MB/s | 475 kB 00:00 2023-07-06T01:16:47.816 INFO:teuthology.orchestra.run.smithi103.stdout:(18/131): python3-ceph-argparse-18.0.0-4742.g79 412 kB/s | 45 kB 00:00 2023-07-06T01:16:47.933 INFO:teuthology.orchestra.run.smithi103.stdout:(19/131): python3-ceph-common-18.0.0-4742.g7936 1.0 MB/s | 119 kB 00:00 2023-07-06T01:16:48.050 INFO:teuthology.orchestra.run.smithi103.stdout:(20/131): python3-cephfs-18.0.0-4742.g79365bb3. 1.6 MB/s | 197 kB 00:00 2023-07-06T01:16:48.175 INFO:teuthology.orchestra.run.smithi103.stdout:(21/131): python3-rados-18.0.0-4742.g79365bb3.e 2.9 MB/s | 370 kB 00:00 2023-07-06T01:16:48.292 INFO:teuthology.orchestra.run.smithi103.stdout:(22/131): python3-rbd-18.0.0-4742.g79365bb3.el8 3.1 MB/s | 364 kB 00:00 2023-07-06T01:16:48.392 INFO:teuthology.orchestra.run.smithi103.stdout:(23/131): python3-rgw-18.0.0-4742.g79365bb3.el8 1.0 MB/s | 107 kB 00:00 2023-07-06T01:16:48.484 INFO:teuthology.orchestra.run.smithi103.stdout:(24/131): rbd-fuse-18.0.0-4742.g79365bb3.el8.x8 1.0 MB/s | 94 kB 00:00 2023-07-06T01:16:48.810 INFO:teuthology.orchestra.run.smithi103.stdout:(25/131): librgw2-18.0.0-4742.g79365bb3.el8.x86 5.9 MB/s | 6.7 MB 00:01 2023-07-06T01:16:48.927 INFO:teuthology.orchestra.run.smithi103.stdout:(26/131): rbd-nbd-18.0.0-4742.g79365bb3.el8.x86 1.5 MB/s | 179 kB 00:00 2023-07-06T01:16:48.993 INFO:teuthology.orchestra.run.smithi103.stdout:(27/131): rbd-mirror-18.0.0-4742.g79365bb3.el8. 6.8 MB/s | 3.5 MB 00:00 2023-07-06T01:16:49.019 INFO:teuthology.orchestra.run.smithi103.stdout:(28/131): ceph-grafana-dashboards-18.0.0-4742.g 251 kB/s | 23 kB 00:00 2023-07-06T01:16:49.078 INFO:teuthology.orchestra.run.smithi103.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4742.g79365bb 1.5 MB/s | 129 kB 00:00 2023-07-06T01:16:49.419 INFO:teuthology.orchestra.run.smithi103.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4742.g79365 4.3 MB/s | 1.7 MB 00:00 2023-07-06T01:16:49.587 INFO:teuthology.orchestra.run.smithi103.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4742.g79 1.4 MB/s | 240 kB 00:00 2023-07-06T01:16:49.687 INFO:teuthology.orchestra.run.smithi103.stdout:(32/131): ceph-mgr-rook-18.0.0-4742.g79365bb3.e 473 kB/s | 47 kB 00:00 2023-07-06T01:16:49.779 INFO:teuthology.orchestra.run.smithi103.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4742.g7 159 kB/s | 15 kB 00:00 2023-07-06T01:16:49.896 INFO:teuthology.orchestra.run.smithi103.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 9.0 MB/s | 7.4 MB 00:00 2023-07-06T01:16:49.937 INFO:teuthology.orchestra.run.smithi103.stdout:(35/131): ceph-volume-18.0.0-4742.g79365bb3.el8 1.6 MB/s | 258 kB 00:00 2023-07-06T01:16:49.963 INFO:teuthology.orchestra.run.smithi103.stdout:(36/131): cephadm-18.0.0-4742.g79365bb3.el8.noa 2.4 MB/s | 166 kB 00:00 2023-07-06T01:16:50.005 INFO:teuthology.orchestra.run.smithi103.stdout:(37/131): cephfs-top-18.0.0-4742.g79365bb3.el8. 268 kB/s | 18 kB 00:00 2023-07-06T01:16:50.147 INFO:teuthology.orchestra.run.smithi103.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 415 kB/s | 59 kB 00:00 2023-07-06T01:16:50.172 INFO:teuthology.orchestra.run.smithi103.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2023-07-06T01:16:50.222 INFO:teuthology.orchestra.run.smithi103.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2023-07-06T01:16:50.256 INFO:teuthology.orchestra.run.smithi103.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 5.6 MB/s | 476 kB 00:00 2023-07-06T01:16:50.281 INFO:teuthology.orchestra.run.smithi103.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 743 kB/s | 44 kB 00:00 2023-07-06T01:16:50.312 INFO:teuthology.orchestra.run.smithi103.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-07-06T01:16:50.316 INFO:teuthology.orchestra.run.smithi022.stdout:(7/131): ceph-common-18.0.0-4742.g79365bb3.el8. 3.8 MB/s | 22 MB 00:05 2023-07-06T01:16:50.332 INFO:teuthology.orchestra.run.smithi103.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 654 kB/s | 33 kB 00:00 2023-07-06T01:16:50.349 INFO:teuthology.orchestra.run.smithi103.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 389 kB/s | 16 kB 00:00 2023-07-06T01:16:50.408 INFO:teuthology.orchestra.run.smithi103.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-07-06T01:16:50.433 INFO:teuthology.orchestra.run.smithi103.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.5 MB/s | 384 kB 00:00 2023-07-06T01:16:50.467 INFO:teuthology.orchestra.run.smithi103.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-07-06T01:16:50.484 INFO:teuthology.orchestra.run.smithi103.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 220 kB/s | 11 kB 00:00 2023-07-06T01:16:50.517 INFO:teuthology.orchestra.run.smithi103.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 351 kB/s | 18 kB 00:00 2023-07-06T01:16:50.591 INFO:teuthology.orchestra.run.smithi022.stdout:(8/131): ceph-mon-18.0.0-4742.g79365bb3.el8.x86 1.5 MB/s | 4.6 MB 00:03 2023-07-06T01:16:50.642 INFO:teuthology.orchestra.run.smithi022.stdout:(9/131): ceph-selinux-18.0.0-4742.g79365bb3.el8 487 kB/s | 24 kB 00:00 2023-07-06T01:16:51.001 INFO:teuthology.orchestra.run.smithi103.stdout:(51/131): ceph-test-18.0.0-4742.g79365bb3.el8.x 11 MB/s | 48 MB 00:04 2023-07-06T01:16:51.018 INFO:teuthology.orchestra.run.smithi103.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 97 kB/s | 49 kB 00:00 2023-07-06T01:16:51.051 INFO:teuthology.orchestra.run.smithi103.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 2.1 MB/s | 1.2 MB 00:00 2023-07-06T01:16:51.077 INFO:teuthology.orchestra.run.smithi103.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-07-06T01:16:51.135 INFO:teuthology.orchestra.run.smithi103.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-07-06T01:16:51.161 INFO:teuthology.orchestra.run.smithi103.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 186 kB/s | 16 kB 00:00 2023-07-06T01:16:51.186 INFO:teuthology.orchestra.run.smithi103.stdout:(57/131): python3-more-itertools-7.2.0-3.el8.no 318 kB/s | 59 kB 00:00 2023-07-06T01:16:51.211 INFO:teuthology.orchestra.run.smithi103.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 445 kB/s | 34 kB 00:00 2023-07-06T01:16:51.237 INFO:teuthology.orchestra.run.smithi103.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2023-07-06T01:16:51.270 INFO:teuthology.orchestra.run.smithi103.stdout:(60/131): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2023-07-06T01:16:51.326 INFO:teuthology.orchestra.run.smithi103.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 164 kB/s | 19 kB 00:00 2023-07-06T01:16:51.354 INFO:teuthology.orchestra.run.smithi103.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 204 kB/s | 24 kB 00:00 2023-07-06T01:16:51.380 INFO:teuthology.orchestra.run.smithi103.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2023-07-06T01:16:51.405 INFO:teuthology.orchestra.run.smithi103.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 350 kB/s | 27 kB 00:00 2023-07-06T01:16:51.430 INFO:teuthology.orchestra.run.smithi103.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2023-07-06T01:16:51.456 INFO:teuthology.orchestra.run.smithi103.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.2 MB/s | 251 kB 00:00 2023-07-06T01:16:51.481 INFO:teuthology.orchestra.run.smithi103.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 812 kB/s | 61 kB 00:00 2023-07-06T01:16:51.506 INFO:teuthology.orchestra.run.smithi103.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-07-06T01:16:51.531 INFO:teuthology.orchestra.run.smithi103.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2023-07-06T01:16:51.556 INFO:teuthology.orchestra.run.smithi103.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-07-06T01:16:51.607 INFO:teuthology.orchestra.run.smithi103.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 625 kB/s | 47 kB 00:00 2023-07-06T01:16:51.632 INFO:teuthology.orchestra.run.smithi103.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.8 MB/s | 200 kB 00:00 2023-07-06T01:16:51.657 INFO:teuthology.orchestra.run.smithi103.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 851 kB/s | 43 kB 00:00 2023-07-06T01:16:51.683 INFO:teuthology.orchestra.run.smithi103.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-06T01:16:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 956 kB/s | 48 kB 00:00 2023-07-06T01:16:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-06T01:16:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 853 kB/s | 43 kB 00:00 2023-07-06T01:16:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-07-06T01:16:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:(79/131): thrift-0.13.0-2.el8.x86_64.rpm 5.0 MB/s | 1.7 MB 00:00 2023-07-06T01:16:51.851 INFO:teuthology.orchestra.run.smithi103.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2023-07-06T01:16:51.859 INFO:teuthology.orchestra.run.smithi022.stdout:(10/131): ceph-radosgw-18.0.0-4742.g79365bb3.el 8.7 MB/s | 13 MB 00:01 2023-07-06T01:16:51.885 INFO:teuthology.orchestra.run.smithi103.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.4 MB/s | 560 kB 00:00 2023-07-06T01:16:51.910 INFO:teuthology.orchestra.run.smithi103.stdout:(82/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 223 kB/s | 19 kB 00:00 2023-07-06T01:16:51.925 INFO:teuthology.orchestra.run.smithi022.stdout:(11/131): cephfs-mirror-18.0.0-4742.g79365bb3.e 3.8 MB/s | 258 kB 00:00 2023-07-06T01:16:51.956 INFO:teuthology.orchestra.run.smithi103.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.5 MB/s | 155 kB 00:00 2023-07-06T01:16:51.984 INFO:teuthology.orchestra.run.smithi022.stdout:(12/131): libcephfs-devel-18.0.0-4742.g79365bb3 533 kB/s | 31 kB 00:00 2023-07-06T01:16:51.994 INFO:teuthology.orchestra.run.smithi103.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 214 kB/s | 18 kB 00:00 2023-07-06T01:16:52.028 INFO:teuthology.orchestra.run.smithi103.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 3.8 MB/s | 559 kB 00:00 2023-07-06T01:16:52.061 INFO:teuthology.orchestra.run.smithi103.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.6 MB/s | 605 kB 00:00 2023-07-06T01:16:52.086 INFO:teuthology.orchestra.run.smithi103.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-07-06T01:16:52.101 INFO:teuthology.orchestra.run.smithi022.stdout:(13/131): libcephfs2-18.0.0-4742.g79365bb3.el8. 6.5 MB/s | 770 kB 00:00 2023-07-06T01:16:52.112 INFO:teuthology.orchestra.run.smithi103.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-07-06T01:16:52.145 INFO:teuthology.orchestra.run.smithi103.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.4 MB/s | 83 kB 00:00 2023-07-06T01:16:52.160 INFO:teuthology.orchestra.run.smithi022.stdout:(14/131): libcephsqlite-18.0.0-4742.g79365bb3.e 2.3 MB/s | 137 kB 00:00 2023-07-06T01:16:52.171 INFO:teuthology.orchestra.run.smithi103.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.0 MB/s | 60 kB 00:00 2023-07-06T01:16:52.218 INFO:teuthology.orchestra.run.smithi022.stdout:(15/131): librados-devel-18.0.0-4742.g79365bb3. 2.3 MB/s | 139 kB 00:00 2023-07-06T01:16:52.273 INFO:teuthology.orchestra.run.smithi103.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 771 kB/s | 98 kB 00:00 2023-07-06T01:16:52.319 INFO:teuthology.orchestra.run.smithi022.stdout:(16/131): libradosstriper1-18.0.0-4742.g79365bb 4.6 MB/s | 475 kB 00:00 2023-07-06T01:16:52.355 INFO:teuthology.orchestra.run.smithi103.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 314 kB/s | 58 kB 00:00 2023-07-06T01:16:52.372 INFO:teuthology.orchestra.run.smithi103.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 392 kB/s | 39 kB 00:00 2023-07-06T01:16:52.430 INFO:teuthology.orchestra.run.smithi103.stdout:(94/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 5.8 MB/s | 2.3 MB 00:00 2023-07-06T01:16:52.456 INFO:teuthology.orchestra.run.smithi103.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.0 MB/s | 171 kB 00:00 2023-07-06T01:16:52.472 INFO:teuthology.orchestra.run.smithi103.stdout:(96/131): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2023-07-06T01:16:52.506 INFO:teuthology.orchestra.run.smithi103.stdout:(97/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.2 MB/s | 644 kB 00:00 2023-07-06T01:16:52.548 INFO:teuthology.orchestra.run.smithi103.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-07-06T01:16:52.623 INFO:teuthology.orchestra.run.smithi103.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2023-07-06T01:16:52.649 INFO:teuthology.orchestra.run.smithi103.stdout:(100/131): python3-pytz-2017.2-9.el8.noarch.rpm 381 kB/s | 54 kB 00:00 2023-07-06T01:16:52.660 INFO:teuthology.orchestra.run.smithi022.stdout:(17/131): ceph-osd-18.0.0-4742.g79365bb3.el8.x8 3.6 MB/s | 18 MB 00:04 2023-07-06T01:16:52.683 INFO:teuthology.orchestra.run.smithi103.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-07-06T01:16:52.711 INFO:teuthology.orchestra.run.smithi022.stdout:(18/131): python3-ceph-argparse-18.0.0-4742.g79 899 kB/s | 45 kB 00:00 2023-07-06T01:16:52.766 INFO:teuthology.orchestra.run.smithi103.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 332 kB/s | 39 kB 00:00 2023-07-06T01:16:52.769 INFO:teuthology.orchestra.run.smithi022.stdout:(19/131): python3-ceph-common-18.0.0-4742.g7936 2.0 MB/s | 119 kB 00:00 2023-07-06T01:16:52.836 INFO:teuthology.orchestra.run.smithi022.stdout:(20/131): python3-cephfs-18.0.0-4742.g79365bb3. 2.9 MB/s | 197 kB 00:00 2023-07-06T01:16:52.908 INFO:teuthology.orchestra.run.smithi103.stdout:(103/131): python3-scipy-1.0.0-21.module+el8.5. 30 MB/s | 14 MB 00:00 2023-07-06T01:16:52.928 INFO:teuthology.orchestra.run.smithi022.stdout:(21/131): python3-rados-18.0.0-4742.g79365bb3.e 3.9 MB/s | 370 kB 00:00 2023-07-06T01:16:52.941 INFO:teuthology.orchestra.run.smithi103.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 3.0 MB/s | 538 kB 00:00 2023-07-06T01:16:52.975 INFO:teuthology.orchestra.run.smithi103.stdout:(105/131): bison-3.0.4-10.el8.x86_64.rpm 2.3 MB/s | 688 kB 00:00 2023-07-06T01:16:53.017 INFO:teuthology.orchestra.run.smithi103.stdout:(106/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.4 MB/s | 157 kB 00:00 2023-07-06T01:16:53.029 INFO:teuthology.orchestra.run.smithi022.stdout:(22/131): python3-rbd-18.0.0-4742.g79365bb3.el8 3.6 MB/s | 364 kB 00:00 2023-07-06T01:16:53.051 INFO:teuthology.orchestra.run.smithi103.stdout:(107/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2023-07-06T01:16:53.087 INFO:teuthology.orchestra.run.smithi022.stdout:(23/131): python3-rgw-18.0.0-4742.g79365bb3.el8 1.8 MB/s | 107 kB 00:00 2023-07-06T01:16:53.146 INFO:teuthology.orchestra.run.smithi022.stdout:(24/131): rbd-fuse-18.0.0-4742.g79365bb3.el8.x8 1.6 MB/s | 94 kB 00:00 2023-07-06T01:16:53.151 INFO:teuthology.orchestra.run.smithi103.stdout:(108/131): automake-1.16.1-7.el8.noarch.rpm 4.0 MB/s | 713 kB 00:00 2023-07-06T01:16:53.230 INFO:teuthology.orchestra.run.smithi103.stdout:(109/131): python3-jsonpatch-1.21-2.el8.noarch. 149 kB/s | 27 kB 00:00 2023-07-06T01:16:53.263 INFO:teuthology.orchestra.run.smithi022.stdout:(25/131): librgw2-18.0.0-4742.g79365bb3.el8.x86 7.1 MB/s | 6.7 MB 00:00 2023-07-06T01:16:53.285 INFO:teuthology.orchestra.run.smithi103.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2023-07-06T01:16:53.353 INFO:teuthology.orchestra.run.smithi022.stdout:(26/131): rbd-nbd-18.0.0-4742.g79365bb3.el8.x86 2.0 MB/s | 179 kB 00:00 2023-07-06T01:16:53.360 INFO:teuthology.orchestra.run.smithi103.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-07-06T01:16:53.377 INFO:teuthology.orchestra.run.smithi103.stdout:(112/131): python3-prettytable-0.7.2-14.el8.noa 298 kB/s | 44 kB 00:00 2023-07-06T01:16:53.413 INFO:teuthology.orchestra.run.smithi022.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4742.g 385 kB/s | 23 kB 00:00 2023-07-06T01:16:53.472 INFO:teuthology.orchestra.run.smithi022.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4742.g79365bb 2.2 MB/s | 129 kB 00:00 2023-07-06T01:16:53.511 INFO:teuthology.orchestra.run.smithi103.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2023-07-06T01:16:53.536 INFO:teuthology.orchestra.run.smithi103.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.4 MB/s | 225 kB 00:00 2023-07-06T01:16:53.561 INFO:teuthology.orchestra.run.smithi103.stdout:(115/131): python3-werkzeug-0.12.2-4.el8.noarch 2.2 MB/s | 457 kB 00:00 2023-07-06T01:16:53.661 INFO:teuthology.orchestra.run.smithi103.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 425 kB/s | 64 kB 00:00 2023-07-06T01:16:53.687 INFO:teuthology.orchestra.run.smithi103.stdout:(117/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 837 kB/s | 126 kB 00:00 2023-07-06T01:16:53.689 INFO:teuthology.orchestra.run.smithi022.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4742.g79365 7.9 MB/s | 1.7 MB 00:00 2023-07-06T01:16:53.764 INFO:teuthology.orchestra.run.smithi022.stdout:(30/131): rbd-mirror-18.0.0-4742.g79365bb3.el8. 5.6 MB/s | 3.5 MB 00:00 2023-07-06T01:16:53.771 INFO:teuthology.orchestra.run.smithi103.stdout:(118/131): python3-jsonpointer-1.10-11.el8.noar 95 kB/s | 20 kB 00:00 2023-07-06T01:16:53.812 INFO:teuthology.orchestra.run.smithi103.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2023-07-06T01:16:53.829 INFO:teuthology.orchestra.run.smithi103.stdout:(120/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 617 kB/s | 103 kB 00:00 2023-07-06T01:16:53.848 INFO:teuthology.orchestra.run.smithi022.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4742.g79 2.8 MB/s | 240 kB 00:00 2023-07-06T01:16:53.963 INFO:teuthology.orchestra.run.smithi103.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 25 MB/s | 4.9 MB 00:00 2023-07-06T01:16:53.973 INFO:teuthology.orchestra.run.smithi022.stdout:(32/131): ceph-mgr-rook-18.0.0-4742.g79365bb3.e 379 kB/s | 47 kB 00:00 2023-07-06T01:16:53.989 INFO:teuthology.orchestra.run.smithi103.stdout:(122/131): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-07-06T01:16:54.089 INFO:teuthology.orchestra.run.smithi103.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-07-06T01:16:54.114 INFO:teuthology.orchestra.run.smithi103.stdout:(124/131): jq-1.6-3.el8.x86_64.rpm 1.3 MB/s | 202 kB 00:00 2023-07-06T01:16:54.132 INFO:teuthology.orchestra.run.smithi022.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4742.g7 91 kB/s | 15 kB 00:00 2023-07-06T01:16:54.147 INFO:teuthology.orchestra.run.smithi103.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2023-07-06T01:16:54.199 INFO:teuthology.orchestra.run.smithi022.stdout:(34/131): ceph-volume-18.0.0-4742.g79365bb3.el8 3.8 MB/s | 258 kB 00:00 2023-07-06T01:16:54.258 INFO:teuthology.orchestra.run.smithi022.stdout:(35/131): cephadm-18.0.0-4742.g79365bb3.el8.noa 2.8 MB/s | 166 kB 00:00 2023-07-06T01:16:54.264 INFO:teuthology.orchestra.run.smithi103.stdout:(126/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-07-06T01:16:54.308 INFO:teuthology.orchestra.run.smithi022.stdout:(36/131): cephfs-top-18.0.0-4742.g79365bb3.el8. 359 kB/s | 18 kB 00:00 2023-07-06T01:16:54.315 INFO:teuthology.orchestra.run.smithi103.stdout:(127/131): postgresql-contrib-10.21-2.module+el 3.9 MB/s | 808 kB 00:00 2023-07-06T01:16:54.390 INFO:teuthology.orchestra.run.smithi103.stdout:(128/131): postgresql-server-10.21-2.module+el8 17 MB/s | 5.1 MB 00:00 2023-07-06T01:16:54.457 INFO:teuthology.orchestra.run.smithi103.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-07-06T01:16:54.732 INFO:teuthology.orchestra.run.smithi103.stdout:(130/131): librbd1-18.0.0-4742.g79365bb3.el8.x8 8.5 MB/s | 3.6 MB 00:00 2023-07-06T01:16:54.767 INFO:teuthology.orchestra.run.smithi022.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 6.9 MB/s | 7.4 MB 00:01 2023-07-06T01:16:54.800 INFO:teuthology.orchestra.run.smithi022.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 623 kB/s | 306 kB 00:00 2023-07-06T01:16:54.892 INFO:teuthology.orchestra.run.smithi022.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 819 kB/s | 75 kB 00:00 2023-07-06T01:16:55.068 INFO:teuthology.orchestra.run.smithi022.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.7 MB/s | 476 kB 00:00 2023-07-06T01:16:55.093 INFO:teuthology.orchestra.run.smithi022.stdout:(41/131): liboath-2.6.2-3.el8.x86_64.rpm 180 kB/s | 59 kB 00:00 2023-07-06T01:16:55.156 INFO:teuthology.orchestra.run.smithi103.stdout:(131/131): librados2-18.0.0-4742.g79365bb3.el8. 4.1 MB/s | 3.7 MB 00:00 2023-07-06T01:16:55.159 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2023-07-06T01:16:55.159 INFO:teuthology.orchestra.run.smithi103.stdout:Total 18 MB/s | 205 MB 00:11 2023-07-06T01:16:55.160 INFO:teuthology.orchestra.run.smithi022.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 475 kB/s | 44 kB 00:00 2023-07-06T01:16:55.252 INFO:teuthology.orchestra.run.smithi022.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 360 kB/s | 33 kB 00:00 2023-07-06T01:16:55.277 INFO:teuthology.orchestra.run.smithi022.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.0 MB/s | 185 kB 00:00 2023-07-06T01:16:55.344 INFO:teuthology.orchestra.run.smithi022.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 177 kB/s | 16 kB 00:00 2023-07-06T01:16:55.378 INFO:teuthology.orchestra.run.smithi022.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-07-06T01:16:55.445 INFO:teuthology.orchestra.run.smithi022.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.7 MB/s | 384 kB 00:00 2023-07-06T01:16:55.470 INFO:teuthology.orchestra.run.smithi022.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 883 kB/s | 81 kB 00:00 2023-07-06T01:16:55.529 INFO:teuthology.orchestra.run.smithi022.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2023-07-06T01:16:55.554 INFO:teuthology.orchestra.run.smithi022.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2023-07-06T01:16:55.638 INFO:teuthology.orchestra.run.smithi022.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 584 kB/s | 49 kB 00:00 2023-07-06T01:16:55.713 INFO:teuthology.orchestra.run.smithi022.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 6.3 MB/s | 1.2 MB 00:00 2023-07-06T01:16:55.730 INFO:teuthology.orchestra.run.smithi022.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 640 kB/s | 59 kB 00:00 2023-07-06T01:16:55.797 INFO:teuthology.orchestra.run.smithi022.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 723 kB/s | 60 kB 00:00 2023-07-06T01:16:55.831 INFO:teuthology.orchestra.run.smithi022.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.8 MB/s | 283 kB 00:00 2023-07-06T01:16:55.881 INFO:teuthology.orchestra.run.smithi022.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 186 kB/s | 16 kB 00:00 2023-07-06T01:16:55.923 INFO:teuthology.orchestra.run.smithi022.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 366 kB/s | 34 kB 00:00 2023-07-06T01:16:55.981 INFO:teuthology.orchestra.run.smithi022.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-07-06T01:16:55.987 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2023-07-06T01:16:56.015 INFO:teuthology.orchestra.run.smithi022.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 667 kB/s | 61 kB 00:00 2023-07-06T01:16:56.074 INFO:teuthology.orchestra.run.smithi022.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 203 kB/s | 19 kB 00:00 2023-07-06T01:16:56.108 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2023-07-06T01:16:56.108 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2023-07-06T01:16:56.108 INFO:teuthology.orchestra.run.smithi022.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 260 kB/s | 24 kB 00:00 2023-07-06T01:16:56.166 INFO:teuthology.orchestra.run.smithi022.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 318 kB/s | 29 kB 00:00 2023-07-06T01:16:56.200 INFO:teuthology.orchestra.run.smithi022.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 299 kB/s | 27 kB 00:00 2023-07-06T01:16:56.267 INFO:teuthology.orchestra.run.smithi022.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-07-06T01:16:56.301 INFO:teuthology.orchestra.run.smithi022.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-07-06T01:16:56.359 INFO:teuthology.orchestra.run.smithi022.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 666 kB/s | 61 kB 00:00 2023-07-06T01:16:56.384 INFO:teuthology.orchestra.run.smithi022.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-07-06T01:16:56.451 INFO:teuthology.orchestra.run.smithi022.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 250 kB/s | 23 kB 00:00 2023-07-06T01:16:56.477 INFO:teuthology.orchestra.run.smithi022.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-06T01:16:56.638 INFO:teuthology.orchestra.run.smithi022.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 6.9 MB/s | 1.7 MB 00:00 2023-07-06T01:16:56.703 INFO:teuthology.orchestra.run.smithi022.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-07-06T01:16:56.748 INFO:teuthology.orchestra.run.smithi022.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-06T01:16:56.786 INFO:teuthology.orchestra.run.smithi022.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-06T01:16:56.828 INFO:teuthology.orchestra.run.smithi022.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-07-06T01:16:56.970 INFO:teuthology.orchestra.run.smithi022.stdout:(75/131): python3-scipy-1.0.0-21.module+el8.5.0 28 MB/s | 14 MB 00:00 2023-07-06T01:16:56.996 INFO:teuthology.orchestra.run.smithi022.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 232 kB/s | 39 kB 00:00 2023-07-06T01:16:57.029 INFO:teuthology.orchestra.run.smithi022.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-07-06T01:16:57.071 INFO:teuthology.orchestra.run.smithi022.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.0 MB/s | 538 kB 00:00 2023-07-06T01:16:57.096 INFO:teuthology.orchestra.run.smithi022.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-07-06T01:16:57.122 INFO:teuthology.orchestra.run.smithi022.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-07-06T01:16:57.155 INFO:teuthology.orchestra.run.smithi022.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-07-06T01:16:57.197 INFO:teuthology.orchestra.run.smithi022.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 636 kB/s | 27 kB 00:00 2023-07-06T01:16:57.216 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2023-07-06T01:16:57.219 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2023-07-06T01:16:57.239 INFO:teuthology.orchestra.run.smithi022.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-06T01:16:57.314 INFO:teuthology.orchestra.run.smithi022.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2023-07-06T01:16:57.332 INFO:teuthology.orchestra.run.smithi022.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 478 kB/s | 44 kB 00:00 2023-07-06T01:16:57.415 INFO:teuthology.orchestra.run.smithi022.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 5.4 MB/s | 457 kB 00:00 2023-07-06T01:16:57.482 INFO:teuthology.orchestra.run.smithi022.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 22 MB/s | 3.7 MB 00:00 2023-07-06T01:16:57.507 INFO:teuthology.orchestra.run.smithi022.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.4 MB/s | 225 kB 00:00 2023-07-06T01:16:57.524 INFO:teuthology.orchestra.run.smithi022.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-07-06T01:16:57.550 INFO:teuthology.orchestra.run.smithi022.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.9 MB/s | 126 kB 00:00 2023-07-06T01:16:57.575 INFO:teuthology.orchestra.run.smithi022.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 397 kB/s | 20 kB 00:00 2023-07-06T01:16:57.600 INFO:teuthology.orchestra.run.smithi022.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-07-06T01:16:57.634 INFO:teuthology.orchestra.run.smithi022.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-06T01:16:57.676 INFO:teuthology.orchestra.run.smithi022.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-06T01:16:57.785 INFO:teuthology.orchestra.run.smithi022.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 26 MB/s | 4.9 MB 00:00 2023-07-06T01:16:57.827 INFO:teuthology.orchestra.run.smithi022.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-07-06T01:16:57.910 INFO:teuthology.orchestra.run.smithi022.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-07-06T01:16:57.935 INFO:teuthology.orchestra.run.smithi022.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-07-06T01:16:57.994 INFO:teuthology.orchestra.run.smithi022.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 14 MB/s | 808 kB 00:00 2023-07-06T01:16:58.103 INFO:teuthology.orchestra.run.smithi022.stdout:(100/131): postgresql-server-10.21-2.module+el8 26 MB/s | 5.1 MB 00:00 2023-07-06T01:16:58.145 INFO:teuthology.orchestra.run.smithi022.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 10 MB/s | 1.5 MB 00:00 2023-07-06T01:16:58.170 INFO:teuthology.orchestra.run.smithi022.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 700 kB/s | 47 kB 00:00 2023-07-06T01:16:58.195 INFO:teuthology.orchestra.run.smithi022.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-07-06T01:16:58.221 INFO:teuthology.orchestra.run.smithi022.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-07-06T01:16:58.246 INFO:teuthology.orchestra.run.smithi022.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-06T01:16:58.663 INFO:teuthology.orchestra.run.smithi022.stdout:(106/131): ceph-test-18.0.0-4742.g79365bb3.el8. 5.9 MB/s | 48 MB 00:08 2023-07-06T01:16:58.689 INFO:teuthology.orchestra.run.smithi022.stdout:(107/131): keyutils-libs-devel-1.5.10-9.el8.x86 103 kB/s | 48 kB 00:00 2023-07-06T01:16:58.722 INFO:teuthology.orchestra.run.smithi022.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 183 kB/s | 87 kB 00:00 2023-07-06T01:16:58.739 INFO:teuthology.orchestra.run.smithi022.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 567 kB/s | 43 kB 00:00 2023-07-06T01:16:58.764 INFO:teuthology.orchestra.run.smithi022.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 1.6 MB/s | 109 kB 00:00 2023-07-06T01:16:58.790 INFO:teuthology.orchestra.run.smithi022.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.3 MB/s | 229 kB 00:00 2023-07-06T01:16:58.806 INFO:teuthology.orchestra.run.smithi022.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 446 kB/s | 19 kB 00:00 2023-07-06T01:16:58.832 INFO:teuthology.orchestra.run.smithi022.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.6 MB/s | 155 kB 00:00 2023-07-06T01:16:58.874 INFO:teuthology.orchestra.run.smithi022.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-07-06T01:16:58.891 INFO:teuthology.orchestra.run.smithi022.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 304 kB/s | 18 kB 00:00 2023-07-06T01:16:58.925 INFO:teuthology.orchestra.run.smithi022.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.6 MB/s | 559 kB 00:00 2023-07-06T01:16:58.958 INFO:teuthology.orchestra.run.smithi022.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-07-06T01:16:58.983 INFO:teuthology.orchestra.run.smithi022.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-07-06T01:16:59.009 INFO:teuthology.orchestra.run.smithi022.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-07-06T01:16:59.034 INFO:teuthology.orchestra.run.smithi022.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-07-06T01:16:59.059 INFO:teuthology.orchestra.run.smithi022.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-07-06T01:16:59.084 INFO:teuthology.orchestra.run.smithi022.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-07-06T01:16:59.110 INFO:teuthology.orchestra.run.smithi022.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-07-06T01:16:59.135 INFO:teuthology.orchestra.run.smithi022.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 770 kB/s | 39 kB 00:00 2023-07-06T01:16:59.177 INFO:teuthology.orchestra.run.smithi022.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-07-06T01:16:59.202 INFO:teuthology.orchestra.run.smithi022.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-07-06T01:16:59.228 INFO:teuthology.orchestra.run.smithi022.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-07-06T01:16:59.278 INFO:teuthology.orchestra.run.smithi022.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.6 MB/s | 2.3 MB 00:00 2023-07-06T01:16:59.336 INFO:teuthology.orchestra.run.smithi022.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-06T01:16:59.648 INFO:teuthology.orchestra.run.smithi022.stdout:(130/131): librados2-18.0.0-4742.g79365bb3.el8. 8.3 MB/s | 3.7 MB 00:00 2023-07-06T01:16:59.829 INFO:teuthology.orchestra.run.smithi022.stdout:(131/131): librbd1-18.0.0-4742.g79365bb3.el8.x8 5.9 MB/s | 3.6 MB 00:00 2023-07-06T01:16:59.832 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2023-07-06T01:16:59.832 INFO:teuthology.orchestra.run.smithi022.stdout:Total 13 MB/s | 205 MB 00:15 2023-07-06T01:17:00.693 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2023-07-06T01:17:00.825 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2023-07-06T01:17:00.825 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2023-07-06T01:17:02.001 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2023-07-06T01:17:02.005 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2023-07-06T01:17:02.264 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2023-07-06T01:17:02.447 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-06T01:17:02.638 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-06T01:17:02.790 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-06T01:17:02.899 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-06T01:17:03.323 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-06T01:17:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-06T01:17:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 5/134 2023-07-06T01:17:04.315 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 5/134 2023-07-06T01:17:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T01:17:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T01:17:04.831 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T01:17:04.913 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T01:17:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T01:17:05.240 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-06T01:17:05.519 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-06T01:17:05.718 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-06T01:17:06.099 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-06T01:17:06.326 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-06T01:17:06.490 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T01:17:06.595 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T01:17:06.781 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T01:17:06.913 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T01:17:07.094 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-06T01:17:07.262 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2023-07-06T01:17:07.287 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-06T01:17:07.419 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-06T01:17:07.487 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-06T01:17:07.563 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-06T01:17:07.614 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-06T01:17:07.756 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-06T01:17:07.763 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-06T01:17:07.881 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-06T01:17:07.929 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-06T01:17:08.058 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-06T01:17:08.279 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-06T01:17:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-06T01:17:08.714 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-06T01:17:08.937 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T01:17:09.198 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-06T01:17:09.344 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T01:17:09.528 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T01:17:09.658 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T01:17:09.780 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-06T01:17:09.828 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T01:17:09.928 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T01:17:09.964 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-06T01:17:10.121 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 9/134 2023-07-06T01:17:10.601 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-06T01:17:10.677 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-06T01:17:10.826 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-06T01:17:10.960 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-06T01:17:11.146 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-06T01:17:11.252 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-06T01:17:11.399 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-06T01:17:11.507 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-06T01:17:11.546 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 26/134 2023-07-06T01:17:11.667 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 27/134 2023-07-06T01:17:11.711 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T01:17:11.818 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 14/134 2023-07-06T01:17:11.905 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 28/134 2023-07-06T01:17:12.003 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T01:17:12.105 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-06T01:17:12.108 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 15/134 2023-07-06T01:17:12.236 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-06T01:17:12.372 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-06T01:17:12.511 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-06T01:17:13.153 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-06T01:17:13.344 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-06T01:17:13.862 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-06T01:17:13.967 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-06T01:17:14.161 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-06T01:17:14.921 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-06T01:17:14.993 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-06T01:17:15.201 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-06T01:17:15.345 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-06T01:17:15.429 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-06T01:17:15.590 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-06T01:17:15.649 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-06T01:17:15.797 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-06T01:17:15.826 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-06T01:17:16.016 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-06T01:17:16.030 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-06T01:17:16.182 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-06T01:17:16.241 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-06T01:17:16.353 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 36/134 2023-07-06T01:17:16.409 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 26/134 2023-07-06T01:17:16.540 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 27/134 2023-07-06T01:17:16.545 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 37/134 2023-07-06T01:17:16.696 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-06T01:17:16.839 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-06T01:17:17.013 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-06T01:17:17.081 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-06T01:17:17.308 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-06T01:17:17.508 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-06T01:17:17.509 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 28/134 2023-07-06T01:17:17.655 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-06T01:17:17.734 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-06T01:17:17.816 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-06T01:17:17.942 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-06T01:17:18.067 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-06T01:17:18.143 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-06T01:17:18.282 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-06T01:17:18.317 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-06T01:17:18.473 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-06T01:17:18.518 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-06T01:17:18.641 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-06T01:17:18.766 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-06T01:17:18.943 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-06T01:17:19.158 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-06T01:17:19.487 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-06T01:17:19.798 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-06T01:17:19.991 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-06T01:17:20.207 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-06T01:17:20.402 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-06T01:17:20.451 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-06T01:17:20.681 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-06T01:17:20.745 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-06T01:17:20.874 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-06T01:17:20.962 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-06T01:17:21.039 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-06T01:17:21.180 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-06T01:17:21.376 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-06T01:17:21.517 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 36/134 2023-07-06T01:17:21.543 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-06T01:17:21.690 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 37/134 2023-07-06T01:17:21.691 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-06T01:17:21.852 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-06T01:17:21.987 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-06T01:17:22.042 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-06T01:17:22.213 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-06T01:17:22.233 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-06T01:17:22.365 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-06T01:17:22.492 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-06T01:17:22.562 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-06T01:17:22.706 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-06T01:17:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-06T01:17:22.854 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-06T01:17:23.040 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-06T01:17:23.129 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-06T01:17:23.296 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-06T01:17:23.414 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-06T01:17:23.502 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-06T01:17:23.613 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-06T01:17:23.641 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-06T01:17:23.806 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-06T01:17:23.839 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-06T01:17:23.978 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-06T01:17:24.084 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-06T01:17:24.148 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-06T01:17:24.321 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-06T01:17:24.404 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-06T01:17:24.490 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-06T01:17:24.634 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-06T01:17:24.651 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-06T01:17:24.845 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-06T01:17:24.847 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-06T01:17:25.085 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-06T01:17:25.091 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-06T01:17:25.258 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-06T01:17:25.267 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-06T01:17:25.423 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-06T01:17:25.501 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-06T01:17:25.723 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-06T01:17:25.736 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-06T01:17:25.901 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-06T01:17:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-06T01:17:26.099 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-06T01:17:26.243 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-06T01:17:26.458 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-06T01:17:26.634 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-06T01:17:26.771 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-06T01:17:26.934 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-06T01:17:27.007 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-06T01:17:27.192 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-06T01:17:27.235 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-06T01:17:27.439 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-06T01:17:27.457 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-06T01:17:27.601 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-06T01:17:27.668 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-06T01:17:27.686 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-06T01:17:27.875 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-06T01:17:27.971 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-06T01:17:28.061 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-06T01:17:28.219 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-06T01:17:28.222 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-06T01:17:28.433 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-06T01:17:28.433 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-06T01:17:28.688 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-06T01:17:28.867 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-06T01:17:29.005 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-06T01:17:29.146 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-06T01:17:29.203 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-06T01:17:29.331 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-06T01:17:29.362 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T01:17:29.425 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T01:17:29.532 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-06T01:17:29.654 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 93/134 2023-07-06T01:17:29.706 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-06T01:17:29.854 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-06T01:17:30.040 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-06T01:17:30.244 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-06T01:17:30.439 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-06T01:17:30.615 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-06T01:17:30.864 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-06T01:17:31.067 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-06T01:17:31.226 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-06T01:17:31.434 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-06T01:17:32.626 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-06T01:17:32.856 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-06T01:17:33.068 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-06T01:17:33.253 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T01:17:33.354 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-06T01:17:33.495 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T01:17:33.591 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-06T01:17:33.790 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-06T01:17:34.091 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T01:17:34.352 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T01:17:34.435 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T01:17:34.603 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T01:17:34.625 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-06T01:17:34.791 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T01:17:34.854 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 92/134 2023-07-06T01:17:35.041 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 93/134 2023-07-06T01:17:38.839 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T01:17:39.091 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T01:17:39.717 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 94/134 2023-07-06T01:17:39.998 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T01:17:40.063 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 95/134 2023-07-06T01:17:40.190 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T01:17:53.985 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T01:17:53.985 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2023-07-06T01:17:53.986 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2023-07-06T01:17:53.986 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /dev 2023-07-06T01:17:53.986 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /run 2023-07-06T01:17:53.986 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2023-07-06T01:17:53.986 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2023-07-06T01:17:53.986 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2023-07-06T01:17:53.986 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2023-07-06T01:17:53.986 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /dev 2023-07-06T01:17:53.987 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:17:54.300 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T01:17:55.873 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T01:17:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T01:17:56.249 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T01:17:56.417 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T01:17:56.869 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T01:17:57.075 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T01:17:57.092 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T01:17:58.790 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T01:17:58.993 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T01:17:59.123 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 102/134 2023-07-06T01:17:59.322 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 103/134 2023-07-06T01:17:59.482 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 96/134 2023-07-06T01:17:59.482 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /sys 2023-07-06T01:17:59.482 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /proc 2023-07-06T01:17:59.483 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /dev 2023-07-06T01:17:59.483 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /run 2023-07-06T01:17:59.483 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /mnt 2023-07-06T01:17:59.483 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /var/tmp 2023-07-06T01:17:59.483 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /home 2023-07-06T01:17:59.483 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /tmp 2023-07-06T01:17:59.483 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /dev 2023-07-06T01:17:59.484 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:17:59.491 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T01:17:59.710 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T01:17:59.812 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T01:17:59.982 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T01:18:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T01:18:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T01:18:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T01:18:01.506 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 97/134 2023-07-06T01:18:01.543 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 107/134 2023-07-06T01:18:01.669 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T01:18:01.732 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T01:18:01.766 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T01:18:01.900 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 98/134 2023-07-06T01:18:01.924 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T01:18:01.935 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T01:18:02.029 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T01:18:02.512 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 99/134 2023-07-06T01:18:02.695 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T01:18:02.713 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 100/134 2023-07-06T01:18:03.219 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 110/134 2023-07-06T01:18:03.468 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T01:18:03.501 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T01:18:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T01:18:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T01:18:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T01:18:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T01:18:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T01:18:04.450 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T01:18:04.638 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 101/134 2023-07-06T01:18:04.768 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 102/134 2023-07-06T01:18:04.955 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 103/134 2023-07-06T01:18:05.153 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T01:18:05.376 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 104/134 2023-07-06T01:18:05.668 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T01:18:06.697 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 105/134 2023-07-06T01:18:06.878 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T01:18:06.975 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 106/134 2023-07-06T01:18:07.348 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 107/134 2023-07-06T01:18:07.635 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T01:18:07.673 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 108/134 2023-07-06T01:18:07.852 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T01:18:07.865 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 109/134 2023-07-06T01:18:08.877 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T01:18:09.239 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 110/134 2023-07-06T01:18:09.484 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T01:18:09.519 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 111/134 2023-07-06T01:18:09.677 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T01:18:09.721 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4742.g79365 112/134 2023-07-06T01:18:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 115/134 2023-07-06T01:18:09.877 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T01:18:10.088 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-06T01:18:10.247 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 113/134 2023-07-06T01:18:10.260 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 117/134 2023-07-06T01:18:10.394 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T01:18:10.534 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-06T01:18:10.689 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-06T01:18:10.883 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-06T01:18:11.144 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-06T01:18:11.294 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 121/134 2023-07-06T01:18:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T01:18:13.543 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T01:18:13.815 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T01:18:13.832 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T01:18:13.990 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 124/134 2023-07-06T01:18:14.281 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 125/134 2023-07-06T01:18:14.569 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 126/134 2023-07-06T01:18:14.905 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-06T01:18:15.122 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-06T01:18:15.208 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-06T01:18:15.241 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 114/134 2023-07-06T01:18:15.389 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-06T01:18:15.402 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-06T01:18:15.564 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-06T01:18:15.572 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-06T01:18:15.648 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 115/134 2023-07-06T01:18:15.699 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-06T01:18:15.700 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T01:18:15.792 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-06T01:18:15.859 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T01:18:15.859 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T01:18:16.005 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-06T01:18:16.061 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T01:18:16.271 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T01:18:16.755 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T01:18:16.755 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 1/134 2023-07-06T01:18:16.755 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2/134 2023-07-06T01:18:16.755 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 3/134 2023-07-06T01:18:16.756 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 4/134 2023-07-06T01:18:16.756 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4742.g79365 5/134 2023-07-06T01:18:16.756 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T01:18:16.756 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T01:18:16.756 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T01:18:16.756 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 9/134 2023-07-06T01:18:16.756 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 10/134 2023-07-06T01:18:16.757 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 11/134 2023-07-06T01:18:16.757 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 12/134 2023-07-06T01:18:16.757 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 13/134 2023-07-06T01:18:16.757 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 14/134 2023-07-06T01:18:16.757 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 15/134 2023-07-06T01:18:16.757 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 16/134 2023-07-06T01:18:16.757 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 17/134 2023-07-06T01:18:16.757 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 18/134 2023-07-06T01:18:16.758 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 19/134 2023-07-06T01:18:16.758 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 20/134 2023-07-06T01:18:16.758 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 21/134 2023-07-06T01:18:16.758 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 22/134 2023-07-06T01:18:16.758 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 23/134 2023-07-06T01:18:16.758 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 24/134 2023-07-06T01:18:16.758 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 25/134 2023-07-06T01:18:16.758 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 26/134 2023-07-06T01:18:16.759 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 27/134 2023-07-06T01:18:16.759 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 28/134 2023-07-06T01:18:16.759 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 29/134 2023-07-06T01:18:16.759 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 30/134 2023-07-06T01:18:16.765 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 31/134 2023-07-06T01:18:16.766 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 32/134 2023-07-06T01:18:16.766 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 33/134 2023-07-06T01:18:16.766 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 34/134 2023-07-06T01:18:16.766 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 35/134 2023-07-06T01:18:16.766 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 36/134 2023-07-06T01:18:16.766 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 37/134 2023-07-06T01:18:16.767 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 38/134 2023-07-06T01:18:16.767 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-06T01:18:16.767 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-06T01:18:16.767 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-06T01:18:16.767 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-06T01:18:16.767 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-06T01:18:16.767 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-06T01:18:16.767 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-06T01:18:16.768 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-06T01:18:16.768 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-06T01:18:16.768 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-06T01:18:16.768 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-06T01:18:16.768 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-06T01:18:16.768 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-06T01:18:16.768 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-06T01:18:16.769 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-06T01:18:16.769 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-06T01:18:16.769 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-06T01:18:16.769 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-06T01:18:16.769 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-06T01:18:16.769 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-06T01:18:16.769 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-06T01:18:16.769 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-06T01:18:16.770 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-06T01:18:16.770 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-06T01:18:16.770 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-06T01:18:16.770 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-06T01:18:16.770 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-06T01:18:16.770 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-06T01:18:16.770 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-06T01:18:16.770 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-06T01:18:16.771 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-06T01:18:16.771 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-06T01:18:16.771 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-06T01:18:16.771 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-06T01:18:16.771 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-06T01:18:16.772 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-06T01:18:16.772 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-06T01:18:16.772 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-06T01:18:16.772 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-06T01:18:16.772 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-06T01:18:16.773 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-06T01:18:16.774 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-06T01:18:16.774 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-06T01:18:16.774 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-06T01:18:16.774 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-06T01:18:16.775 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-06T01:18:16.775 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-06T01:18:16.775 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-06T01:18:16.775 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-06T01:18:16.775 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-06T01:18:16.776 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-06T01:18:16.776 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-06T01:18:16.776 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-06T01:18:16.776 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-06T01:18:16.776 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-06T01:18:16.776 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-06T01:18:16.776 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-06T01:18:16.776 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-06T01:18:16.777 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-06T01:18:16.777 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-06T01:18:16.777 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-06T01:18:16.777 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-06T01:18:16.777 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-06T01:18:16.777 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-06T01:18:16.777 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-06T01:18:16.778 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-06T01:18:16.778 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-06T01:18:16.778 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-06T01:18:16.778 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-06T01:18:16.778 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-06T01:18:16.778 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-06T01:18:16.778 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-06T01:18:16.778 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-06T01:18:16.779 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-06T01:18:16.779 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-06T01:18:16.779 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-06T01:18:16.779 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-06T01:18:16.779 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-06T01:18:16.779 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-06T01:18:16.779 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-06T01:18:16.779 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-06T01:18:16.780 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-06T01:18:16.780 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-06T01:18:16.780 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-06T01:18:16.780 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-06T01:18:16.780 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-06T01:18:16.780 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-06T01:18:16.780 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-06T01:18:16.781 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-06T01:18:16.781 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-06T01:18:16.781 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 129/134 2023-07-06T01:18:16.781 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-06T01:18:16.781 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 131/134 2023-07-06T01:18:16.781 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T01:18:16.781 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-06T01:18:16.955 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-06T01:18:17.387 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-06T01:18:17.566 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 119/134 2023-07-06T01:18:17.738 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-06T01:18:17.922 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 121/134 2023-07-06T01:18:18.069 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T01:18:19.444 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T01:18:19.445 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:01:48 ago on Thu 06 Jul 2023 01:16:31 AM UTC. 2023-07-06T01:18:20.338 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T01:18:20.567 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T01:18:20.587 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T01:18:20.843 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 124/134 2023-07-06T01:18:21.101 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 125/134 2023-07-06T01:18:21.389 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 126/134 2023-07-06T01:18:21.693 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-06T01:18:21.858 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T01:18:21.868 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T01:18:22.049 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-06T01:18:22.162 INFO:teuthology.orchestra.run.smithi103.stdout:Installed products updated. 2023-07-06T01:18:22.216 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-06T01:18:22.301 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-06T01:18:22.400 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:22.400 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2023-07-06T01:18:22.400 INFO:teuthology.orchestra.run.smithi103.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-06T01:18:22.400 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.400 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.401 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2023-07-06T01:18:22.401 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-06T01:18:22.401 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.1-7.el8.noarch 2023-07-06T01:18:22.401 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.0.4-10.el8.x86_64 2023-07-06T01:18:22.401 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.401 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.401 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.402 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.402 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:22.402 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.402 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.402 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.402 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:22.402 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:22.403 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:22.403 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:22.403 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:22.403 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.403 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.403 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:22.403 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.403 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.404 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.404 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:22.404 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:22.404 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.404 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:22.404 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-06T01:18:22.404 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.1-9.el8.x86_64 2023-07-06T01:18:22.404 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-06T01:18:22.405 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-06T01:18:22.405 INFO:teuthology.orchestra.run.smithi103.stdout: jq-1.6-3.el8.x86_64 2023-07-06T01:18:22.405 INFO:teuthology.orchestra.run.smithi103.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-06T01:18:22.405 INFO:teuthology.orchestra.run.smithi103.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-06T01:18:22.405 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-06T01:18:22.405 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.405 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.405 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.406 INFO:teuthology.orchestra.run.smithi103.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-06T01:18:22.406 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-06T01:18:22.406 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-06T01:18:22.406 INFO:teuthology.orchestra.run.smithi103.stdout: libpq-13.5-1.el8.x86_64 2023-07-06T01:18:22.406 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-06T01:18:22.406 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-06T01:18:22.406 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.407 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.407 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-06T01:18:22.407 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.407 INFO:teuthology.orchestra.run.smithi103.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-06T01:18:22.407 INFO:teuthology.orchestra.run.smithi103.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-06T01:18:22.407 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-06T01:18:22.407 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-06T01:18:22.408 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-06T01:18:22.408 INFO:teuthology.orchestra.run.smithi103.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-06T01:18:22.408 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-06T01:18:22.408 INFO:teuthology.orchestra.run.smithi103.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-06T01:18:22.408 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-06T01:18:22.408 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-06T01:18:22.408 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-06T01:18:22.408 INFO:teuthology.orchestra.run.smithi103.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-06T01:18:22.409 INFO:teuthology.orchestra.run.smithi103.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-06T01:18:22.409 INFO:teuthology.orchestra.run.smithi103.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-06T01:18:22.409 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T01:18:22.409 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T01:18:22.409 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T01:18:22.409 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-06T01:18:22.409 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-06T01:18:22.409 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-06T01:18:22.410 INFO:teuthology.orchestra.run.smithi103.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-06T01:18:22.410 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-06T01:18:22.410 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.410 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.410 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.410 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-06T01:18:22.410 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-06T01:18:22.410 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-06T01:18:22.411 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-06T01:18:22.411 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-06T01:18:22.411 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-06T01:18:22.411 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-06T01:18:22.411 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-06T01:18:22.411 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-06T01:18:22.411 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-06T01:18:22.411 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-06T01:18:22.412 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-06T01:18:22.412 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-06T01:18:22.412 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-06T01:18:22.412 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-06T01:18:22.412 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-06T01:18:22.412 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-06T01:18:22.412 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-06T01:18:22.413 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-06T01:18:22.413 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-06T01:18:22.413 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-06T01:18:22.413 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-06T01:18:22.413 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-2.6-1.el8.noarch 2023-07-06T01:18:22.413 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-06T01:18:22.413 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-06T01:18:22.413 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-06T01:18:22.414 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-06T01:18:22.414 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-06T01:18:22.414 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-06T01:18:22.414 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.414 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.414 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-06T01:18:22.414 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-06T01:18:22.415 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.415 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-06T01:18:22.415 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-06T01:18:22.415 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-06T01:18:22.415 INFO:teuthology.orchestra.run.smithi103.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-06T01:18:22.415 INFO:teuthology.orchestra.run.smithi103.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-06T01:18:22.415 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-06T01:18:22.415 INFO:teuthology.orchestra.run.smithi103.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-06T01:18:22.416 INFO:teuthology.orchestra.run.smithi103.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-06T01:18:22.416 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-06T01:18:22.416 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-06T01:18:22.416 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-06T01:18:22.416 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-06T01:18:22.416 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-06T01:18:22.416 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.416 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.417 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:22.417 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-06T01:18:22.417 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-06T01:18:22.417 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-06T01:18:22.417 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-06T01:18:22.417 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-06T01:18:22.418 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-06T01:18:22.418 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:22.418 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2023-07-06T01:18:22.533 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-06T01:18:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-06T01:18:22.547 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T01:18:22.582 DEBUG:teuthology.parallel:result is None 2023-07-06T01:18:22.720 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T01:18:22.720 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T01:18:22.939 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T01:18:23.107 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T01:18:23.621 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T01:18:23.621 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 1/134 2023-07-06T01:18:23.621 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2/134 2023-07-06T01:18:23.622 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 3/134 2023-07-06T01:18:23.622 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 4/134 2023-07-06T01:18:23.622 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4742.g79365 5/134 2023-07-06T01:18:23.622 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 6/134 2023-07-06T01:18:23.622 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 7/134 2023-07-06T01:18:23.622 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 8/134 2023-07-06T01:18:23.622 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 9/134 2023-07-06T01:18:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 10/134 2023-07-06T01:18:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 11/134 2023-07-06T01:18:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 12/134 2023-07-06T01:18:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 13/134 2023-07-06T01:18:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_ 14/134 2023-07-06T01:18:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 15/134 2023-07-06T01:18:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 16/134 2023-07-06T01:18:23.623 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_6 17/134 2023-07-06T01:18:23.624 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86 18/134 2023-07-06T01:18:23.624 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 19/134 2023-07-06T01:18:23.624 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el 20/134 2023-07-06T01:18:23.624 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-common-2:18.0.0-4742.g79365bb3.el8. 21/134 2023-07-06T01:18:23.624 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_6 22/134 2023-07-06T01:18:23.624 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 23/134 2023-07-06T01:18:23.624 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 24/134 2023-07-06T01:18:23.625 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 25/134 2023-07-06T01:18:23.625 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 26/134 2023-07-06T01:18:23.625 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 27/134 2023-07-06T01:18:23.625 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 28/134 2023-07-06T01:18:23.625 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3. 29/134 2023-07-06T01:18:23.632 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noa 30/134 2023-07-06T01:18:23.632 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.n 31/134 2023-07-06T01:18:23.632 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4742.g793 32/134 2023-07-06T01:18:23.632 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el 33/134 2023-07-06T01:18:23.633 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 34/134 2023-07-06T01:18:23.633 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.e 35/134 2023-07-06T01:18:23.633 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 36/134 2023-07-06T01:18:23.633 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 37/134 2023-07-06T01:18:23.633 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 38/134 2023-07-06T01:18:23.633 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-06T01:18:23.633 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-06T01:18:23.633 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-06T01:18:23.634 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-06T01:18:23.634 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-06T01:18:23.634 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-06T01:18:23.634 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-06T01:18:23.634 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-06T01:18:23.634 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-06T01:18:23.634 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-06T01:18:23.634 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-06T01:18:23.635 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-06T01:18:23.635 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-06T01:18:23.635 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-06T01:18:23.635 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-06T01:18:23.635 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-06T01:18:23.635 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-06T01:18:23.635 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-06T01:18:23.636 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-06T01:18:23.636 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-06T01:18:23.636 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-06T01:18:23.636 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-06T01:18:23.636 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-06T01:18:23.636 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-06T01:18:23.636 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-06T01:18:23.636 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-06T01:18:23.637 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-06T01:18:23.637 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-06T01:18:23.637 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-06T01:18:23.637 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-06T01:18:23.637 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-06T01:18:23.637 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-06T01:18:23.637 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-06T01:18:23.637 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-06T01:18:23.638 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-06T01:18:23.638 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-06T01:18:23.638 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-06T01:18:23.638 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-06T01:18:23.638 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-06T01:18:23.638 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-06T01:18:23.638 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-06T01:18:23.638 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-06T01:18:23.639 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-06T01:18:23.639 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-06T01:18:23.639 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-06T01:18:23.639 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-06T01:18:23.639 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-06T01:18:23.639 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-06T01:18:23.639 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-06T01:18:23.640 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-06T01:18:23.640 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-06T01:18:23.640 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-06T01:18:23.640 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-06T01:18:23.640 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-06T01:18:23.640 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-06T01:18:23.640 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-06T01:18:23.640 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-06T01:18:23.641 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-06T01:18:23.641 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-06T01:18:23.641 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-06T01:18:23.641 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-06T01:18:23.641 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-06T01:18:23.641 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-06T01:18:23.641 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-06T01:18:23.641 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-06T01:18:23.642 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-06T01:18:23.642 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-06T01:18:23.642 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-06T01:18:23.642 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-06T01:18:23.642 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-06T01:18:23.642 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-06T01:18:23.642 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-06T01:18:23.642 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-06T01:18:23.643 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-06T01:18:23.643 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-06T01:18:23.643 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-06T01:18:23.643 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-06T01:18:23.643 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-06T01:18:23.643 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-06T01:18:23.643 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-06T01:18:23.644 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-06T01:18:23.644 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-06T01:18:23.644 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-06T01:18:23.644 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T01:18:23.644 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-06T01:18:23.644 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-06T01:18:23.644 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-06T01:18:23.644 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-06T01:18:23.645 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-06T01:18:23.645 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T01:18:23.645 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 129/134 2023-07-06T01:18:23.645 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-06T01:18:23.645 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 131/134 2023-07-06T01:18:23.645 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T01:18:23.645 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-06T01:18:26.321 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T01:18:26.321 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:54 ago on Thu 06 Jul 2023 01:16:32 AM UTC. 2023-07-06T01:18:29.342 INFO:teuthology.orchestra.run.smithi022.stdout:Installed products updated. 2023-07-06T01:18:29.594 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:29.594 INFO:teuthology.orchestra.run.smithi022.stdout:Upgraded: 2023-07-06T01:18:29.595 INFO:teuthology.orchestra.run.smithi022.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-06T01:18:29.595 INFO:teuthology.orchestra.run.smithi022.stdout: librados2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.595 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.596 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2023-07-06T01:18:29.596 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-06T01:18:29.596 INFO:teuthology.orchestra.run.smithi022.stdout: automake-1.16.1-7.el8.noarch 2023-07-06T01:18:29.596 INFO:teuthology.orchestra.run.smithi022.stdout: bison-3.0.4-10.el8.x86_64 2023-07-06T01:18:29.596 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.597 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.597 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.597 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.597 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:29.597 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.597 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.597 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.598 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:29.598 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:29.598 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:29.598 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:29.598 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:29.598 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.598 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.598 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:29.599 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.599 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.599 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.599 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:29.600 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:29.600 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.600 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top-2:18.0.0-4742.g79365bb3.el8.noarch 2023-07-06T01:18:29.601 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-06T01:18:29.601 INFO:teuthology.orchestra.run.smithi022.stdout: flex-2.6.1-9.el8.x86_64 2023-07-06T01:18:29.601 INFO:teuthology.orchestra.run.smithi022.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-06T01:18:29.601 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-06T01:18:29.602 INFO:teuthology.orchestra.run.smithi022.stdout: jq-1.6-3.el8.x86_64 2023-07-06T01:18:29.602 INFO:teuthology.orchestra.run.smithi022.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-06T01:18:29.602 INFO:teuthology.orchestra.run.smithi022.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-06T01:18:29.602 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-06T01:18:29.603 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.603 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.603 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.604 INFO:teuthology.orchestra.run.smithi022.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-06T01:18:29.604 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-06T01:18:29.604 INFO:teuthology.orchestra.run.smithi022.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-06T01:18:29.604 INFO:teuthology.orchestra.run.smithi022.stdout: libpq-13.5-1.el8.x86_64 2023-07-06T01:18:29.605 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-06T01:18:29.605 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-06T01:18:29.605 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.605 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.605 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-06T01:18:29.605 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.606 INFO:teuthology.orchestra.run.smithi022.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-06T01:18:29.606 INFO:teuthology.orchestra.run.smithi022.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-06T01:18:29.606 INFO:teuthology.orchestra.run.smithi022.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-06T01:18:29.606 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-06T01:18:29.606 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-06T01:18:29.606 INFO:teuthology.orchestra.run.smithi022.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-06T01:18:29.606 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-06T01:18:29.607 INFO:teuthology.orchestra.run.smithi022.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-06T01:18:29.607 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-06T01:18:29.607 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-06T01:18:29.607 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-06T01:18:29.607 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-06T01:18:29.607 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-06T01:18:29.607 INFO:teuthology.orchestra.run.smithi022.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-06T01:18:29.607 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T01:18:29.608 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T01:18:29.608 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T01:18:29.608 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-06T01:18:29.608 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-06T01:18:29.609 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-06T01:18:29.609 INFO:teuthology.orchestra.run.smithi022.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-06T01:18:29.609 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-06T01:18:29.609 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.610 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.610 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.610 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-06T01:18:29.610 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-06T01:18:29.611 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-06T01:18:29.611 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-06T01:18:29.611 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-06T01:18:29.611 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-06T01:18:29.612 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-06T01:18:29.612 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-06T01:18:29.612 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-06T01:18:29.613 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-06T01:18:29.613 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-06T01:18:29.613 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-06T01:18:29.613 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-06T01:18:29.613 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-06T01:18:29.614 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-06T01:18:29.614 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-06T01:18:29.614 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-06T01:18:29.614 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-06T01:18:29.614 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-06T01:18:29.615 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-06T01:18:29.615 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-06T01:18:29.615 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-06T01:18:29.615 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend-2.6-1.el8.noarch 2023-07-06T01:18:29.616 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-06T01:18:29.616 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-06T01:18:29.616 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-06T01:18:29.616 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-06T01:18:29.616 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-06T01:18:29.616 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-06T01:18:29.617 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.617 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.617 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-06T01:18:29.617 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-06T01:18:29.617 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.617 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-06T01:18:29.618 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-06T01:18:29.618 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-06T01:18:29.618 INFO:teuthology.orchestra.run.smithi022.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-06T01:18:29.618 INFO:teuthology.orchestra.run.smithi022.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-06T01:18:29.618 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-06T01:18:29.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-06T01:18:29.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-06T01:18:29.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-06T01:18:29.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-06T01:18:29.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-06T01:18:29.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-06T01:18:29.619 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-06T01:18:29.620 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.620 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.620 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd-2:18.0.0-4742.g79365bb3.el8.x86_64 2023-07-06T01:18:29.620 INFO:teuthology.orchestra.run.smithi022.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-06T01:18:29.620 INFO:teuthology.orchestra.run.smithi022.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-06T01:18:29.620 INFO:teuthology.orchestra.run.smithi022.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-06T01:18:29.620 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-06T01:18:29.620 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-06T01:18:29.621 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-06T01:18:29.621 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:29.621 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2023-07-06T01:18:29.785 DEBUG:teuthology.parallel:result is None 2023-07-06T01:18:29.786 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T01:18:29.930 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-06T01:18:29.954 INFO:teuthology.orchestra.run.smithi022.stdout:18.0.0-4742.g79365bb3.el8 2023-07-06T01:18:29.954 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4742.g79365bb3.el8 2023-07-06T01:18:29.955 INFO:teuthology.task.install:The correct ceph version 18.0.0-4742.g79365bb3 is installed. 2023-07-06T01:18:29.955 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=79365bb3b803780276d69efb2cc4afe5b160de9e 2023-07-06T01:18:30.173 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-06T01:18:30.201 INFO:teuthology.orchestra.run.smithi103.stdout:18.0.0-4742.g79365bb3.el8 2023-07-06T01:18:30.202 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4742.g79365bb3.el8 2023-07-06T01:18:30.202 INFO:teuthology.task.install:The correct ceph version 18.0.0-4742.g79365bb3 is installed. 2023-07-06T01:18:30.203 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-06T01:18:30.204 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:18:30.204 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-06T01:18:30.247 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:18:30.247 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-06T01:18:30.293 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-06T01:18:30.293 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:18:30.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2023-07-06T01:18:30.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-06T01:18:30.389 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:18:30.389 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2023-07-06T01:18:30.418 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-06T01:18:30.486 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-06T01:18:30.487 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:18:30.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-06T01:18:30.514 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-06T01:18:30.581 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:18:30.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-06T01:18:30.610 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-06T01:18:30.676 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-06T01:18:30.677 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:18:30.677 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2023-07-06T01:18:30.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-06T01:18:30.772 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:18:30.772 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2023-07-06T01:18:30.800 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-06T01:18:30.866 INFO:teuthology.run_tasks:Running task ceph... 2023-07-06T01:18:30.937 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-07-06T01:18:30.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /var/log/ceph 2023-07-06T01:18:30.940 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /var/log/ceph 2023-07-06T01:18:30.966 INFO:tasks.ceph:Disabling ceph logrotate... 2023-07-06T01:18:30.968 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-06T01:18:31.007 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-06T01:18:31.033 INFO:tasks.ceph:Creating extra log directories... 2023-07-06T01:18:31.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-06T01:18:31.074 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-06T01:18:31.105 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-07-06T01:18:31.106 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-07-06T01:18:31.106 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-07-06T01:18:31.106 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:18:31.106 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-06T01:18:31.156 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-06T01:18:31.294 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-06T01:18:31.321 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:18:31.321 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-06T01:18:31.338 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-06T01:18:31.467 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-06T01:18:31.498 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-07-06T01:18:31.499 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-07-06T01:18:31.499 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789', 'branch': 'wip-rishabh-2023Jul5', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub}', 'email': None, 'first_in_suite': False, 'job_id': '7327789', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-2023Jul5'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-2023Jul5', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-2023Jul5', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi022.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi103.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '79365bb3b803780276d69efb2cc4afe5b160de9e'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b53f236c1077f1d295baafb9e0ca9a4b447019d7', 'timestamp': '2023-07-05_22:21:20', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-07-06T01:18:31.500 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-06T01:18:31.502 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-06T01:18:31.550 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/run/ceph 2023-07-06T01:18:31.559 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m0777 -- /var/run/ceph 2023-07-06T01:18:31.617 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:18:31.618 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2023-07-06T01:18:31.643 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-06T01:18:31.643 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2023-07-06T01:18:31.701 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-06T01:18:31.701 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T01:18:31.701 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 6h/6d Inode: 383140 Links: 1 2023-07-06T01:18:31.701 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T01:18:31.701 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T01:18:31.702 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2023-07-06 01:18:23.305977910 +0000 2023-07-06T01:18:31.702 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2023-07-06 01:13:56.144259342 +0000 2023-07-06T01:18:31.702 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2023-07-06 01:13:56.144259342 +0000 2023-07-06T01:18:31.702 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2023-07-06T01:18:31.703 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-06T01:18:31.767 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2023-07-06T01:18:31.767 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2023-07-06T01:18:31.767 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000105399 s, 4.9 MB/s 2023-07-06T01:18:31.768 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-06T01:18:31.826 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2023-07-06T01:18:31.884 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-06T01:18:31.884 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T01:18:31.884 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 6h/6d Inode: 379247 Links: 1 2023-07-06T01:18:31.884 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T01:18:31.884 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T01:18:31.885 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2023-07-06 01:18:23.305977910 +0000 2023-07-06T01:18:31.885 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2023-07-06 01:13:56.837249784 +0000 2023-07-06T01:18:31.885 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2023-07-06 01:13:56.837249784 +0000 2023-07-06T01:18:31.885 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2023-07-06T01:18:31.885 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-06T01:18:31.949 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2023-07-06T01:18:31.950 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2023-07-06T01:18:31.950 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000110416 s, 4.6 MB/s 2023-07-06T01:18:31.951 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-06T01:18:32.009 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2023-07-06T01:18:32.067 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-06T01:18:32.067 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T01:18:32.068 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 6h/6d Inode: 375549 Links: 1 2023-07-06T01:18:32.068 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T01:18:32.068 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T01:18:32.069 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2023-07-06 01:18:23.305977910 +0000 2023-07-06T01:18:32.069 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2023-07-06 01:13:57.378242323 +0000 2023-07-06T01:18:32.069 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2023-07-06 01:13:57.378242323 +0000 2023-07-06T01:18:32.069 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2023-07-06T01:18:32.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-06T01:18:32.132 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2023-07-06T01:18:32.133 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2023-07-06T01:18:32.133 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000111787 s, 4.6 MB/s 2023-07-06T01:18:32.134 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-06T01:18:32.192 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2023-07-06T01:18:32.250 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-06T01:18:32.250 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T01:18:32.250 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 6h/6d Inode: 384296 Links: 1 2023-07-06T01:18:32.250 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T01:18:32.251 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T01:18:32.251 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2023-07-06 01:18:23.305977910 +0000 2023-07-06T01:18:32.251 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2023-07-06 01:13:57.905235054 +0000 2023-07-06T01:18:32.251 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2023-07-06 01:13:57.905235054 +0000 2023-07-06T01:18:32.251 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: - 2023-07-06T01:18:32.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-06T01:18:32.315 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2023-07-06T01:18:32.316 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2023-07-06T01:18:32.316 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000103109 s, 5.0 MB/s 2023-07-06T01:18:32.317 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-06T01:18:32.375 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T01:18:32.375 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:18:32.376 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2023-07-06T01:18:32.394 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-06T01:18:32.394 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2023-07-06T01:18:32.451 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-06T01:18:32.452 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T01:18:32.452 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 371703 Links: 1 2023-07-06T01:18:32.452 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T01:18:32.452 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T01:18:32.452 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2023-07-06 01:18:16.457008564 +0000 2023-07-06T01:18:32.452 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2023-07-06 01:15:42.614901499 +0000 2023-07-06T01:18:32.453 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2023-07-06 01:15:42.614901499 +0000 2023-07-06T01:18:32.453 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: - 2023-07-06T01:18:32.453 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-06T01:18:32.518 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2023-07-06T01:18:32.518 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2023-07-06T01:18:32.518 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000258406 s, 2.0 MB/s 2023-07-06T01:18:32.519 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-06T01:18:32.578 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2023-07-06T01:18:32.635 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-06T01:18:32.635 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T01:18:32.635 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 382114 Links: 1 2023-07-06T01:18:32.635 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T01:18:32.635 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T01:18:32.636 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2023-07-06 01:18:16.457008564 +0000 2023-07-06T01:18:32.636 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2023-07-06 01:15:43.176890937 +0000 2023-07-06T01:18:32.636 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2023-07-06 01:15:43.176890937 +0000 2023-07-06T01:18:32.636 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: - 2023-07-06T01:18:32.636 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-06T01:18:32.707 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2023-07-06T01:18:32.707 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2023-07-06T01:18:32.708 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000243108 s, 2.1 MB/s 2023-07-06T01:18:32.708 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-06T01:18:32.765 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2023-07-06T01:18:32.822 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-06T01:18:32.823 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T01:18:32.823 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 379194 Links: 1 2023-07-06T01:18:32.823 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T01:18:32.823 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T01:18:32.823 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2023-07-06 01:18:16.457008564 +0000 2023-07-06T01:18:32.823 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2023-07-06 01:15:43.695881183 +0000 2023-07-06T01:18:32.823 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2023-07-06 01:15:43.695881183 +0000 2023-07-06T01:18:32.824 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: - 2023-07-06T01:18:32.824 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-06T01:18:32.887 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2023-07-06T01:18:32.888 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2023-07-06T01:18:32.888 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000237935 s, 2.2 MB/s 2023-07-06T01:18:32.889 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-06T01:18:32.946 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2023-07-06T01:18:33.003 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-06T01:18:33.004 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T01:18:33.004 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 377647 Links: 1 2023-07-06T01:18:33.004 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T01:18:33.004 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T01:18:33.004 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2023-07-06 01:18:16.457008564 +0000 2023-07-06T01:18:33.004 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2023-07-06 01:15:44.224871241 +0000 2023-07-06T01:18:33.005 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2023-07-06 01:15:44.224871241 +0000 2023-07-06T01:18:33.005 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: - 2023-07-06T01:18:33.005 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-06T01:18:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2023-07-06T01:18:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2023-07-06T01:18:33.068 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000208823 s, 2.5 MB/s 2023-07-06T01:18:33.070 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-06T01:18:33.128 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T01:18:33.128 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi022.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi103.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-06T01:18:33.129 INFO:tasks.ceph:Generating config... 2023-07-06T01:18:33.130 INFO:tasks.ceph:[client] client mount timeout = 600 2023-07-06T01:18:33.130 INFO:tasks.ceph:[client] debug client = 20 2023-07-06T01:18:33.130 INFO:tasks.ceph:[client] debug ms = 1 2023-07-06T01:18:33.131 INFO:tasks.ceph:[client] pid file = 2023-07-06T01:18:33.131 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-07-06T01:18:33.131 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-07-06T01:18:33.131 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-07-06T01:18:33.131 INFO:tasks.ceph:[mds] debug mds = 20 2023-07-06T01:18:33.132 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-07-06T01:18:33.132 INFO:tasks.ceph:[mds] debug ms = 1 2023-07-06T01:18:33.132 INFO:tasks.ceph:[mds] mds cache max size = 1000 2023-07-06T01:18:33.132 INFO:tasks.ceph:[mds] mds debug frag = True 2023-07-06T01:18:33.132 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-07-06T01:18:33.133 INFO:tasks.ceph:[mds] mds log max segments = 1 2023-07-06T01:18:33.133 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-07-06T01:18:33.133 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-07-06T01:18:33.133 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-07-06T01:18:33.133 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-07-06T01:18:33.134 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-07-06T01:18:33.134 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-07-06T01:18:33.134 INFO:tasks.ceph:[mgr] debug ms = 1 2023-07-06T01:18:33.134 INFO:tasks.ceph:[mon] debug mon = 20 2023-07-06T01:18:33.134 INFO:tasks.ceph:[mon] debug ms = 1 2023-07-06T01:18:33.134 INFO:tasks.ceph:[mon] debug paxos = 20 2023-07-06T01:18:33.135 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-07-06T01:18:33.135 INFO:tasks.ceph:[osd] bdev async discard = True 2023-07-06T01:18:33.135 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-07-06T01:18:33.135 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-07-06T01:18:33.135 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-07-06T01:18:33.136 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-07-06T01:18:33.136 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-07-06T01:18:33.136 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-07-06T01:18:33.136 INFO:tasks.ceph:[osd] debug ms = 1 2023-07-06T01:18:33.136 INFO:tasks.ceph:[osd] debug osd = 20 2023-07-06T01:18:33.137 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-07-06T01:18:33.137 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-07-06T01:18:33.137 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-07-06T01:18:33.137 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-07-06T01:18:33.137 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-07-06T01:18:33.138 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-07-06T01:18:33.138 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-07-06T01:18:33.138 INFO:tasks.ceph:Setting up mon.a... 2023-07-06T01:18:33.138 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-07-06T01:18:33.201 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.keyring 2023-07-06T01:18:33.204 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-07-06T01:18:33.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T01:18:33.284 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.22'), ('mon.b', '172.21.15.103'), ('mon.c', '[v2:172.21.15.103:3301,v1:172.21.15.103:6790]')] 2023-07-06T01:18:33.285 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.22,172.21.15.103,[v2:172.21.15.103:3301,v1:172.21.15.103:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-07-06T01:18:33.286 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:18:33.286 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-07-06T01:18:33.342 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.22 --add b 172.21.15.103 --addv c '[v2:172.21.15.103:3301,v1:172.21.15.103:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:18:33.434 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:18:33.434 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: generated fsid 63a894dd-9157-4ce0-adf3-c1bdfe16ec64 2023-07-06T01:18:33.434 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = pacific 2023-07-06T01:18:33.434 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 0 2023-07-06T01:18:33.435 INFO:teuthology.orchestra.run.smithi022.stdout:fsid 63a894dd-9157-4ce0-adf3-c1bdfe16ec64 2023-07-06T01:18:33.435 INFO:teuthology.orchestra.run.smithi022.stdout:last_changed 2023-07-06T01:18:33.433069+0000 2023-07-06T01:18:33.435 INFO:teuthology.orchestra.run.smithi022.stdout:created 2023-07-06T01:18:33.433069+0000 2023-07-06T01:18:33.435 INFO:teuthology.orchestra.run.smithi022.stdout:min_mon_release 16 (pacific) 2023-07-06T01:18:33.435 INFO:teuthology.orchestra.run.smithi022.stdout:election_strategy: 1 2023-07-06T01:18:33.435 INFO:teuthology.orchestra.run.smithi022.stdout:0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2023-07-06T01:18:33.435 INFO:teuthology.orchestra.run.smithi022.stdout:1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2023-07-06T01:18:33.436 INFO:teuthology.orchestra.run.smithi022.stdout:2: [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] mon.c 2023-07-06T01:18:33.436 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-07-06T01:18:33.437 DEBUG:teuthology.orchestra.run.smithi022:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-07-06T01:18:33.455 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 63a894dd-9157-4ce0-adf3-c1bdfe16ec64... 2023-07-06T01:18:33.456 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T01:18:33.498 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T01:18:33.543 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2023-07-06T01:18:33.544 INFO:teuthology.orchestra.run.smithi022.stdout: chdir = "" 2023-07-06T01:18:33.544 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T01:18:33.544 INFO:teuthology.orchestra.run.smithi022.stdout: auth supported = cephx 2023-07-06T01:18:33.544 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.544 INFO:teuthology.orchestra.run.smithi022.stdout: filestore xattr use omap = true 2023-07-06T01:18:33.545 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.545 INFO:teuthology.orchestra.run.smithi022.stdout: mon clock drift allowed = 1.000 2023-07-06T01:18:33.545 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.545 INFO:teuthology.orchestra.run.smithi022.stdout: osd crush chooseleaf type = 0 2023-07-06T01:18:33.545 INFO:teuthology.orchestra.run.smithi022.stdout: auth debug = true 2023-07-06T01:18:33.545 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.545 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on old message = true 2023-07-06T01:18:33.545 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on bug = true 2023-07-06T01:18:33.546 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.546 INFO:teuthology.orchestra.run.smithi022.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T01:18:33.546 INFO:teuthology.orchestra.run.smithi022.stdout: mon pg warn max object skew = 0 2023-07-06T01:18:33.546 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.546 INFO:teuthology.orchestra.run.smithi022.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T01:18:33.546 INFO:teuthology.orchestra.run.smithi022.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T01:18:33.546 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.547 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default size = 2 2023-07-06T01:18:33.547 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.547 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow primary affinity = true 2023-07-06T01:18:33.547 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow pg remap = true 2023-07-06T01:18:33.547 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on legacy crush tunables = false 2023-07-06T01:18:33.547 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on crush straw calc version zero = false 2023-07-06T01:18:33.547 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on no sortbitwise = false 2023-07-06T01:18:33.547 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on osd down out interval zero = false 2023-07-06T01:18:33.548 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on too few osds = false 2023-07-06T01:18:33.548 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T01:18:33.548 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T01:18:33.548 INFO:teuthology.orchestra.run.smithi022.stdout: mon_allow_pool_size_one = true 2023-07-06T01:18:33.548 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.549 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T01:18:33.549 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.549 INFO:teuthology.orchestra.run.smithi022.stdout: osd default data pool replay window = 5 2023-07-06T01:18:33.549 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.549 INFO:teuthology.orchestra.run.smithi022.stdout: mon allow pool delete = true 2023-07-06T01:18:33.549 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.549 INFO:teuthology.orchestra.run.smithi022.stdout: mon cluster log file level = debug 2023-07-06T01:18:33.549 INFO:teuthology.orchestra.run.smithi022.stdout: debug asserts on shutdown = true 2023-07-06T01:18:33.550 INFO:teuthology.orchestra.run.smithi022.stdout: mon health detail to clog = false 2023-07-06T01:18:33.550 INFO:teuthology.orchestra.run.smithi022.stdout: mon host = "172.21.15.22,172.21.15.103,[v2:172.21.15.103:3301,v1:172.21.15.103:6790]" 2023-07-06T01:18:33.550 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on skipped message = False 2023-07-06T01:18:33.550 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = 63a894dd-9157-4ce0-adf3-c1bdfe16ec64 2023-07-06T01:18:33.550 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.550 INFO:teuthology.orchestra.run.smithi022.stdout:[osd] 2023-07-06T01:18:33.550 INFO:teuthology.orchestra.run.smithi022.stdout: osd journal size = 100 2023-07-06T01:18:33.551 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.551 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub load threshold = 5.0 2023-07-06T01:18:33.551 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub max interval = 600 2023-07-06T01:18:33.551 INFO:teuthology.orchestra.run.smithi022.stdout: osd mclock profile = high_recovery_ops 2023-07-06T01:18:33.551 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.551 INFO:teuthology.orchestra.run.smithi022.stdout: osd recover clone overlap = true 2023-07-06T01:18:33.551 INFO:teuthology.orchestra.run.smithi022.stdout: osd recovery max chunk = 1048576 2023-07-06T01:18:33.552 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.552 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug shutdown = true 2023-07-06T01:18:33.552 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug op order = true 2023-07-06T01:18:33.552 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug verify stray on activate = true 2023-07-06T01:18:33.552 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.552 INFO:teuthology.orchestra.run.smithi022.stdout: osd open classes on start = true 2023-07-06T01:18:33.552 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug pg log writeout = true 2023-07-06T01:18:33.552 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.553 INFO:teuthology.orchestra.run.smithi022.stdout: osd deep scrub update digest min age = 30 2023-07-06T01:18:33.553 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.553 INFO:teuthology.orchestra.run.smithi022.stdout: osd map max advance = 10 2023-07-06T01:18:33.553 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.553 INFO:teuthology.orchestra.run.smithi022.stdout: journal zero on create = true 2023-07-06T01:18:33.554 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.554 INFO:teuthology.orchestra.run.smithi022.stdout: filestore ondisk finisher threads = 3 2023-07-06T01:18:33.554 INFO:teuthology.orchestra.run.smithi022.stdout: filestore apply finisher threads = 3 2023-07-06T01:18:33.554 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.554 INFO:teuthology.orchestra.run.smithi022.stdout: bdev debug aio = true 2023-07-06T01:18:33.554 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug misdirected ops = true 2023-07-06T01:18:33.554 INFO:teuthology.orchestra.run.smithi022.stdout: bdev async discard = True 2023-07-06T01:18:33.554 INFO:teuthology.orchestra.run.smithi022.stdout: bdev enable discard = True 2023-07-06T01:18:33.555 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore allocator = bitmap 2023-07-06T01:18:33.555 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore block size = 96636764160 2023-07-06T01:18:33.555 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore fsck on mount = True 2023-07-06T01:18:33.555 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluefs = 1/20 2023-07-06T01:18:33.555 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluestore = 1/20 2023-07-06T01:18:33.555 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-07-06T01:18:33.555 INFO:teuthology.orchestra.run.smithi022.stdout: debug osd = 20 2023-07-06T01:18:33.556 INFO:teuthology.orchestra.run.smithi022.stdout: debug rocksdb = 4/10 2023-07-06T01:18:33.556 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T01:18:33.556 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd full ratio = 0.9 2023-07-06T01:18:33.556 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd nearfull ratio = 0.8 2023-07-06T01:18:33.556 INFO:teuthology.orchestra.run.smithi022.stdout: osd failsafe full ratio = 0.95 2023-07-06T01:18:33.556 INFO:teuthology.orchestra.run.smithi022.stdout: osd objectstore = bluestore 2023-07-06T01:18:33.556 INFO:teuthology.orchestra.run.smithi022.stdout: osd op complaint time = 180 2023-07-06T01:18:33.557 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.557 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr] 2023-07-06T01:18:33.557 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-07-06T01:18:33.557 INFO:teuthology.orchestra.run.smithi022.stdout: debug mgr = 20 2023-07-06T01:18:33.557 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2023-07-06T01:18:33.557 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2023-07-06T01:18:33.557 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2023-07-06T01:18:33.557 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2023-07-06T01:18:33.558 INFO:teuthology.orchestra.run.smithi022.stdout: mgr/telemetry/nag = false 2023-07-06T01:18:33.558 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.558 INFO:teuthology.orchestra.run.smithi022.stdout:[mon] 2023-07-06T01:18:33.558 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-07-06T01:18:33.558 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2023-07-06T01:18:33.558 INFO:teuthology.orchestra.run.smithi022.stdout: debug paxos = 20 2023-07-06T01:18:33.558 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2023-07-06T01:18:33.558 INFO:teuthology.orchestra.run.smithi022.stdout: mon data avail warn = 5 2023-07-06T01:18:33.559 INFO:teuthology.orchestra.run.smithi022.stdout: mon mgr mkfs grace = 240 2023-07-06T01:18:33.559 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2023-07-06T01:18:33.559 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd reporter subtree level = osd 2023-07-06T01:18:33.559 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd prime pg temp = true 2023-07-06T01:18:33.559 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2023-07-06T01:18:33.559 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.559 INFO:teuthology.orchestra.run.smithi022.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T01:18:33.560 INFO:teuthology.orchestra.run.smithi022.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T01:18:33.560 INFO:teuthology.orchestra.run.smithi022.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T01:18:33.560 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.560 INFO:teuthology.orchestra.run.smithi022.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T01:18:33.560 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T01:18:33.560 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T01:18:33.560 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.560 INFO:teuthology.orchestra.run.smithi022.stdout: # 1m isn't quite enough 2023-07-06T01:18:33.561 INFO:teuthology.orchestra.run.smithi022.stdout: mon_down_mkfs_grace = 2m 2023-07-06T01:18:33.561 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.561 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_filestore_osds = false 2023-07-06T01:18:33.561 INFO:teuthology.orchestra.run.smithi022.stdout: mon op complaint time = 120 2023-07-06T01:18:33.561 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:18:33.561 INFO:teuthology.orchestra.run.smithi022.stdout:[client] 2023-07-06T01:18:33.561 INFO:teuthology.orchestra.run.smithi022.stdout: rgw cache enabled = true 2023-07-06T01:18:33.561 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable ops log = true 2023-07-06T01:18:33.562 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable usage log = true 2023-07-06T01:18:33.562 INFO:teuthology.orchestra.run.smithi022.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T01:18:33.562 INFO:teuthology.orchestra.run.smithi022.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T01:18:33.562 INFO:teuthology.orchestra.run.smithi022.stdout: client mount timeout = 600 2023-07-06T01:18:33.562 INFO:teuthology.orchestra.run.smithi022.stdout: debug client = 20 2023-07-06T01:18:33.562 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-07-06T01:18:33.562 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = "" 2023-07-06T01:18:33.563 INFO:teuthology.orchestra.run.smithi022.stdout: rados mon op timeout = 900 2023-07-06T01:18:33.563 INFO:teuthology.orchestra.run.smithi022.stdout: rados osd op timeout = 900 2023-07-06T01:18:33.563 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.a] 2023-07-06T01:18:33.563 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.b] 2023-07-06T01:18:33.563 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.c] 2023-07-06T01:18:33.563 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2023-07-06T01:18:33.563 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2023-07-06T01:18:33.563 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2023-07-06T01:18:33.564 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2023-07-06T01:18:33.564 INFO:teuthology.orchestra.run.smithi022.stdout:[mds] 2023-07-06T01:18:33.564 INFO:teuthology.orchestra.run.smithi022.stdout: debug mds = 20 2023-07-06T01:18:33.564 INFO:teuthology.orchestra.run.smithi022.stdout: debug mds balancer = 20 2023-07-06T01:18:33.564 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2023-07-06T01:18:33.564 INFO:teuthology.orchestra.run.smithi022.stdout: mds cache max size = 1000 2023-07-06T01:18:33.564 INFO:teuthology.orchestra.run.smithi022.stdout: mds debug frag = True 2023-07-06T01:18:33.565 INFO:teuthology.orchestra.run.smithi022.stdout: mds debug scatterstat = True 2023-07-06T01:18:33.565 INFO:teuthology.orchestra.run.smithi022.stdout: mds log max segments = 1 2023-07-06T01:18:33.565 INFO:teuthology.orchestra.run.smithi022.stdout: mds op complaint time = 180 2023-07-06T01:18:33.565 INFO:teuthology.orchestra.run.smithi022.stdout: mds verify scatter = True 2023-07-06T01:18:33.565 INFO:teuthology.orchestra.run.smithi022.stdout: osd op complaint time = 180 2023-07-06T01:18:33.565 INFO:teuthology.orchestra.run.smithi022.stdout: rados mon op timeout = 900 2023-07-06T01:18:33.565 INFO:teuthology.orchestra.run.smithi022.stdout: rados osd op timeout = 900 2023-07-06T01:18:33.567 INFO:teuthology.orchestra.run.smithi103.stdout:[global] 2023-07-06T01:18:33.567 INFO:teuthology.orchestra.run.smithi103.stdout: chdir = "" 2023-07-06T01:18:33.567 INFO:teuthology.orchestra.run.smithi103.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T01:18:33.567 INFO:teuthology.orchestra.run.smithi103.stdout: auth supported = cephx 2023-07-06T01:18:33.567 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.568 INFO:teuthology.orchestra.run.smithi103.stdout: filestore xattr use omap = true 2023-07-06T01:18:33.568 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.568 INFO:teuthology.orchestra.run.smithi103.stdout: mon clock drift allowed = 1.000 2023-07-06T01:18:33.568 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.568 INFO:teuthology.orchestra.run.smithi103.stdout: osd crush chooseleaf type = 0 2023-07-06T01:18:33.568 INFO:teuthology.orchestra.run.smithi103.stdout: auth debug = true 2023-07-06T01:18:33.568 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.569 INFO:teuthology.orchestra.run.smithi103.stdout: ms die on old message = true 2023-07-06T01:18:33.569 INFO:teuthology.orchestra.run.smithi103.stdout: ms die on bug = true 2023-07-06T01:18:33.569 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.569 INFO:teuthology.orchestra.run.smithi103.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T01:18:33.569 INFO:teuthology.orchestra.run.smithi103.stdout: mon pg warn max object skew = 0 2023-07-06T01:18:33.569 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.569 INFO:teuthology.orchestra.run.smithi103.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T01:18:33.570 INFO:teuthology.orchestra.run.smithi103.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T01:18:33.570 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.570 INFO:teuthology.orchestra.run.smithi103.stdout: osd pool default size = 2 2023-07-06T01:18:33.570 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.570 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd allow primary affinity = true 2023-07-06T01:18:33.570 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd allow pg remap = true 2023-07-06T01:18:33.570 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on legacy crush tunables = false 2023-07-06T01:18:33.570 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on crush straw calc version zero = false 2023-07-06T01:18:33.571 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on no sortbitwise = false 2023-07-06T01:18:33.571 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on osd down out interval zero = false 2023-07-06T01:18:33.571 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on too few osds = false 2023-07-06T01:18:33.571 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T01:18:33.571 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T01:18:33.571 INFO:teuthology.orchestra.run.smithi103.stdout: mon_allow_pool_size_one = true 2023-07-06T01:18:33.571 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.571 INFO:teuthology.orchestra.run.smithi103.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T01:18:33.572 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.572 INFO:teuthology.orchestra.run.smithi103.stdout: osd default data pool replay window = 5 2023-07-06T01:18:33.572 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.572 INFO:teuthology.orchestra.run.smithi103.stdout: mon allow pool delete = true 2023-07-06T01:18:33.572 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.572 INFO:teuthology.orchestra.run.smithi103.stdout: mon cluster log file level = debug 2023-07-06T01:18:33.572 INFO:teuthology.orchestra.run.smithi103.stdout: debug asserts on shutdown = true 2023-07-06T01:18:33.573 INFO:teuthology.orchestra.run.smithi103.stdout: mon health detail to clog = false 2023-07-06T01:18:33.573 INFO:teuthology.orchestra.run.smithi103.stdout: mon host = "172.21.15.22,172.21.15.103,[v2:172.21.15.103:3301,v1:172.21.15.103:6790]" 2023-07-06T01:18:33.573 INFO:teuthology.orchestra.run.smithi103.stdout: ms die on skipped message = False 2023-07-06T01:18:33.573 INFO:teuthology.orchestra.run.smithi103.stdout: fsid = 63a894dd-9157-4ce0-adf3-c1bdfe16ec64 2023-07-06T01:18:33.573 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.573 INFO:teuthology.orchestra.run.smithi103.stdout:[osd] 2023-07-06T01:18:33.573 INFO:teuthology.orchestra.run.smithi103.stdout: osd journal size = 100 2023-07-06T01:18:33.573 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.574 INFO:teuthology.orchestra.run.smithi103.stdout: osd scrub load threshold = 5.0 2023-07-06T01:18:33.574 INFO:teuthology.orchestra.run.smithi103.stdout: osd scrub max interval = 600 2023-07-06T01:18:33.574 INFO:teuthology.orchestra.run.smithi103.stdout: osd mclock profile = high_recovery_ops 2023-07-06T01:18:33.574 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.574 INFO:teuthology.orchestra.run.smithi103.stdout: osd recover clone overlap = true 2023-07-06T01:18:33.574 INFO:teuthology.orchestra.run.smithi103.stdout: osd recovery max chunk = 1048576 2023-07-06T01:18:33.574 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.575 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug shutdown = true 2023-07-06T01:18:33.575 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug op order = true 2023-07-06T01:18:33.575 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug verify stray on activate = true 2023-07-06T01:18:33.575 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.575 INFO:teuthology.orchestra.run.smithi103.stdout: osd open classes on start = true 2023-07-06T01:18:33.575 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug pg log writeout = true 2023-07-06T01:18:33.575 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.575 INFO:teuthology.orchestra.run.smithi103.stdout: osd deep scrub update digest min age = 30 2023-07-06T01:18:33.576 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.576 INFO:teuthology.orchestra.run.smithi103.stdout: osd map max advance = 10 2023-07-06T01:18:33.576 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.576 INFO:teuthology.orchestra.run.smithi103.stdout: journal zero on create = true 2023-07-06T01:18:33.576 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.576 INFO:teuthology.orchestra.run.smithi103.stdout: filestore ondisk finisher threads = 3 2023-07-06T01:18:33.576 INFO:teuthology.orchestra.run.smithi103.stdout: filestore apply finisher threads = 3 2023-07-06T01:18:33.577 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.577 INFO:teuthology.orchestra.run.smithi103.stdout: bdev debug aio = true 2023-07-06T01:18:33.577 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug misdirected ops = true 2023-07-06T01:18:33.577 INFO:teuthology.orchestra.run.smithi103.stdout: bdev async discard = True 2023-07-06T01:18:33.577 INFO:teuthology.orchestra.run.smithi103.stdout: bdev enable discard = True 2023-07-06T01:18:33.577 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore allocator = bitmap 2023-07-06T01:18:33.577 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore block size = 96636764160 2023-07-06T01:18:33.577 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore fsck on mount = True 2023-07-06T01:18:33.578 INFO:teuthology.orchestra.run.smithi103.stdout: debug bluefs = 1/20 2023-07-06T01:18:33.578 INFO:teuthology.orchestra.run.smithi103.stdout: debug bluestore = 1/20 2023-07-06T01:18:33.578 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2023-07-06T01:18:33.578 INFO:teuthology.orchestra.run.smithi103.stdout: debug osd = 20 2023-07-06T01:18:33.578 INFO:teuthology.orchestra.run.smithi103.stdout: debug rocksdb = 4/10 2023-07-06T01:18:33.578 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T01:18:33.578 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd full ratio = 0.9 2023-07-06T01:18:33.578 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd nearfull ratio = 0.8 2023-07-06T01:18:33.579 INFO:teuthology.orchestra.run.smithi103.stdout: osd failsafe full ratio = 0.95 2023-07-06T01:18:33.579 INFO:teuthology.orchestra.run.smithi103.stdout: osd objectstore = bluestore 2023-07-06T01:18:33.579 INFO:teuthology.orchestra.run.smithi103.stdout: osd op complaint time = 180 2023-07-06T01:18:33.579 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.579 INFO:teuthology.orchestra.run.smithi103.stdout:[mgr] 2023-07-06T01:18:33.579 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2023-07-06T01:18:33.579 INFO:teuthology.orchestra.run.smithi103.stdout: debug mgr = 20 2023-07-06T01:18:33.579 INFO:teuthology.orchestra.run.smithi103.stdout: debug mon = 20 2023-07-06T01:18:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: debug auth = 20 2023-07-06T01:18:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min pgs per osd = 4 2023-07-06T01:18:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min bytes per osd = 10 2023-07-06T01:18:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: mgr/telemetry/nag = false 2023-07-06T01:18:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.580 INFO:teuthology.orchestra.run.smithi103.stdout:[mon] 2023-07-06T01:18:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2023-07-06T01:18:33.580 INFO:teuthology.orchestra.run.smithi103.stdout: debug mon = 20 2023-07-06T01:18:33.581 INFO:teuthology.orchestra.run.smithi103.stdout: debug paxos = 20 2023-07-06T01:18:33.581 INFO:teuthology.orchestra.run.smithi103.stdout: debug auth = 20 2023-07-06T01:18:33.581 INFO:teuthology.orchestra.run.smithi103.stdout: mon data avail warn = 5 2023-07-06T01:18:33.581 INFO:teuthology.orchestra.run.smithi103.stdout: mon mgr mkfs grace = 240 2023-07-06T01:18:33.581 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min pgs per osd = 4 2023-07-06T01:18:33.581 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd reporter subtree level = osd 2023-07-06T01:18:33.581 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd prime pg temp = true 2023-07-06T01:18:33.582 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min bytes per osd = 10 2023-07-06T01:18:33.582 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.582 INFO:teuthology.orchestra.run.smithi103.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T01:18:33.582 INFO:teuthology.orchestra.run.smithi103.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T01:18:33.582 INFO:teuthology.orchestra.run.smithi103.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T01:18:33.582 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.582 INFO:teuthology.orchestra.run.smithi103.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T01:18:33.582 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T01:18:33.583 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T01:18:33.583 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.583 INFO:teuthology.orchestra.run.smithi103.stdout: # 1m isn't quite enough 2023-07-06T01:18:33.583 INFO:teuthology.orchestra.run.smithi103.stdout: mon_down_mkfs_grace = 2m 2023-07-06T01:18:33.583 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.583 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_filestore_osds = false 2023-07-06T01:18:33.583 INFO:teuthology.orchestra.run.smithi103.stdout: mon op complaint time = 120 2023-07-06T01:18:33.583 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-07-06T01:18:33.584 INFO:teuthology.orchestra.run.smithi103.stdout:[client] 2023-07-06T01:18:33.584 INFO:teuthology.orchestra.run.smithi103.stdout: rgw cache enabled = true 2023-07-06T01:18:33.584 INFO:teuthology.orchestra.run.smithi103.stdout: rgw enable ops log = true 2023-07-06T01:18:33.584 INFO:teuthology.orchestra.run.smithi103.stdout: rgw enable usage log = true 2023-07-06T01:18:33.584 INFO:teuthology.orchestra.run.smithi103.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T01:18:33.584 INFO:teuthology.orchestra.run.smithi103.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T01:18:33.584 INFO:teuthology.orchestra.run.smithi103.stdout: client mount timeout = 600 2023-07-06T01:18:33.584 INFO:teuthology.orchestra.run.smithi103.stdout: debug client = 20 2023-07-06T01:18:33.585 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2023-07-06T01:18:33.585 INFO:teuthology.orchestra.run.smithi103.stdout: pid file = "" 2023-07-06T01:18:33.585 INFO:teuthology.orchestra.run.smithi103.stdout: rados mon op timeout = 900 2023-07-06T01:18:33.585 INFO:teuthology.orchestra.run.smithi103.stdout: rados osd op timeout = 900 2023-07-06T01:18:33.585 INFO:teuthology.orchestra.run.smithi103.stdout:[mon.a] 2023-07-06T01:18:33.585 INFO:teuthology.orchestra.run.smithi103.stdout:[mon.b] 2023-07-06T01:18:33.586 INFO:teuthology.orchestra.run.smithi103.stdout:[mon.c] 2023-07-06T01:18:33.586 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.a] 2023-07-06T01:18:33.586 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.c] 2023-07-06T01:18:33.586 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.b] 2023-07-06T01:18:33.586 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.d] 2023-07-06T01:18:33.586 INFO:teuthology.orchestra.run.smithi103.stdout:[mds] 2023-07-06T01:18:33.586 INFO:teuthology.orchestra.run.smithi103.stdout: debug mds = 20 2023-07-06T01:18:33.586 INFO:teuthology.orchestra.run.smithi103.stdout: debug mds balancer = 20 2023-07-06T01:18:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2023-07-06T01:18:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: mds cache max size = 1000 2023-07-06T01:18:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: mds debug frag = True 2023-07-06T01:18:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: mds debug scatterstat = True 2023-07-06T01:18:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: mds log max segments = 1 2023-07-06T01:18:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: mds op complaint time = 180 2023-07-06T01:18:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: mds verify scatter = True 2023-07-06T01:18:33.587 INFO:teuthology.orchestra.run.smithi103.stdout: osd op complaint time = 180 2023-07-06T01:18:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: rados mon op timeout = 900 2023-07-06T01:18:33.588 INFO:teuthology.orchestra.run.smithi103.stdout: rados osd op timeout = 900 2023-07-06T01:18:33.588 INFO:tasks.ceph:Creating admin key on mon.a... 2023-07-06T01:18:33.588 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-07-06T01:18:33.666 INFO:tasks.ceph:Copying monmap to all nodes... 2023-07-06T01:18:33.666 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:18:33.667 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-07-06T01:18:33.683 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:18:33.684 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-07-06T01:18:33.740 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:18:33.740 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:18:33.741 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-06T01:18:33.741 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T01:18:33.818 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:18:33.818 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:18:33.875 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi103.front.sepia.ceph.com 2023-07-06T01:18:33.876 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:18:33.876 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-06T01:18:33.876 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T01:18:33.913 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:18:33.914 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:18:33.970 INFO:tasks.ceph:Setting up mon nodes... 2023-07-06T01:18:33.971 INFO:tasks.ceph:Setting up mgr nodes... 2023-07-06T01:18:33.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-07-06T01:18:34.027 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-07-06T01:18:34.029 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-07-06T01:18:34.102 INFO:teuthology.orchestra.run.smithi103.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-07-06T01:18:34.105 INFO:tasks.ceph:Setting up mds nodes... 2023-07-06T01:18:34.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-07-06T01:18:34.160 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-07-06T01:18:34.163 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-07-06T01:18:34.190 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-07-06T01:18:34.283 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-07-06T01:18:34.286 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-07-06T01:18:34.312 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-07-06T01:18:34.367 INFO:teuthology.orchestra.run.smithi103.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-07-06T01:18:34.370 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-07-06T01:18:34.396 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-07-06T01:18:34.489 INFO:teuthology.orchestra.run.smithi103.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-07-06T01:18:34.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-07-06T01:18:34.517 INFO:tasks.ceph_client:Setting up client nodes... 2023-07-06T01:18:34.517 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-06T01:18:34.562 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-07-06T01:18:34.575 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-07-06T01:18:34.619 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-07-06T01:18:34.699 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-07-06T01:18:34.745 INFO:teuthology.orchestra.run.smithi103.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-07-06T01:18:34.757 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-07-06T01:18:34.800 INFO:teuthology.orchestra.run.smithi103.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-07-06T01:18:34.822 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-07-06T01:18:34.823 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi022.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi103.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-06T01:18:34.823 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-07-06T01:18:34.852 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T01:18:34.852 INFO:tasks.ceph:role: osd.0 2023-07-06T01:18:34.853 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:18:34.853 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-06T01:18:34.920 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T01:18:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T01:18:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T01:18:34.921 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T01:18:34.922 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T01:18:34.922 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2023-07-06T01:18:34.922 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T01:18:34.923 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T01:18:34.923 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T01:18:34.923 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T01:18:34.943 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2023-07-06T01:18:34.944 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2023-07-06T01:18:34.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-07-06T01:18:35.026 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-07-06T01:18:35.096 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-07-06T01:18:35.163 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T01:18:35.164 INFO:tasks.ceph:role: osd.1 2023-07-06T01:18:35.164 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:18:35.164 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-06T01:18:35.230 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T01:18:35.230 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T01:18:35.231 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T01:18:35.231 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T01:18:35.231 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T01:18:35.231 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2023-07-06T01:18:35.231 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T01:18:35.231 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T01:18:35.232 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T01:18:35.232 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T01:18:35.253 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2023-07-06T01:18:35.255 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2023-07-06T01:18:35.255 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-07-06T01:18:35.335 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-07-06T01:18:35.406 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-07-06T01:18:35.473 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T01:18:35.473 INFO:tasks.ceph:role: osd.2 2023-07-06T01:18:35.474 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:18:35.474 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-06T01:18:35.539 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T01:18:35.540 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T01:18:35.540 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T01:18:35.540 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T01:18:35.540 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T01:18:35.540 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2023-07-06T01:18:35.541 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T01:18:35.541 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T01:18:35.541 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T01:18:35.541 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T01:18:35.562 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2023-07-06T01:18:35.563 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2023-07-06T01:18:35.564 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-07-06T01:18:35.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-07-06T01:18:35.716 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-07-06T01:18:35.783 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T01:18:35.783 INFO:tasks.ceph:role: osd.3 2023-07-06T01:18:35.784 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:18:35.784 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-06T01:18:35.849 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T01:18:35.850 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T01:18:35.850 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T01:18:35.850 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T01:18:35.850 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T01:18:35.850 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2023-07-06T01:18:35.850 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T01:18:35.851 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T01:18:35.851 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T01:18:35.851 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T01:18:35.872 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2023-07-06T01:18:35.873 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2023-07-06T01:18:35.874 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-07-06T01:18:35.953 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-07-06T01:18:36.024 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:18:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:36.323+0000 7f09d249a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-07-06T01:18:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:36.323+0000 7f09d249a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-07-06T01:18:36.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:36.323+0000 7f09d249a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T01:18:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:36.323+0000 7f09d249a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T01:18:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:36.323+0000 7f09d249a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T01:18:36.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:36.323+0000 7f09d249a640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-07-06T01:18:39.189 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-07-06T01:18:39.221 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:18:39.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:39.313+0000 7f371ad2a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-07-06T01:18:39.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:39.313+0000 7f371ad2a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-07-06T01:18:39.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:39.313+0000 7f371ad2a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T01:18:39.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:39.313+0000 7f371ad2a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T01:18:39.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:39.313+0000 7f371ad2a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T01:18:39.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:39.314+0000 7f371ad2a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-07-06T01:18:42.188 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-07-06T01:18:42.220 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:18:42.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:42.353+0000 7fb2bcef8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-07-06T01:18:42.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:42.353+0000 7fb2bcef8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-07-06T01:18:42.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:42.353+0000 7fb2bcef8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T01:18:42.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:42.353+0000 7fb2bcef8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T01:18:42.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:42.353+0000 7fb2bcef8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T01:18:42.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:42.354+0000 7fb2bcef8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-07-06T01:18:45.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-07-06T01:18:45.267 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:18:45.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:45.358+0000 7fc7c1620640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-07-06T01:18:45.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:45.358+0000 7fc7c1620640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-07-06T01:18:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:45.358+0000 7fc7c1620640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T01:18:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:45.358+0000 7fc7c1620640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T01:18:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:45.358+0000 7fc7c1620640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T01:18:45.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:18:45.358+0000 7fc7c1620640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-07-06T01:18:48.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-07-06T01:18:48.256 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-07-06T01:18:48.290 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T01:18:48.290 INFO:tasks.ceph:role: osd.4 2023-07-06T01:18:48.291 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi103.front.sepia.ceph.com 2023-07-06T01:18:48.291 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-06T01:18:48.368 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T01:18:48.368 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T01:18:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T01:18:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T01:18:48.369 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T01:18:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2023-07-06T01:18:48.369 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T01:18:48.369 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T01:18:48.369 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T01:18:48.370 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T01:18:48.793 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2023-07-06T01:18:48.796 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2023-07-06T01:18:48.797 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-07-06T01:18:48.974 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-07-06T01:18:49.012 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-07-06T01:18:49.082 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T01:18:49.082 INFO:tasks.ceph:role: osd.5 2023-07-06T01:18:49.083 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi103.front.sepia.ceph.com 2023-07-06T01:18:49.083 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-06T01:18:49.158 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T01:18:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T01:18:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T01:18:49.159 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T01:18:49.159 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T01:18:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2023-07-06T01:18:49.160 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T01:18:49.160 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T01:18:49.160 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T01:18:49.160 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T01:18:49.584 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2023-07-06T01:18:49.586 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2023-07-06T01:18:49.587 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-07-06T01:18:49.701 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-07-06T01:18:49.735 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-07-06T01:18:49.802 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T01:18:49.803 INFO:tasks.ceph:role: osd.6 2023-07-06T01:18:49.803 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi103.front.sepia.ceph.com 2023-07-06T01:18:49.803 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-06T01:18:49.877 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T01:18:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T01:18:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T01:18:49.878 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T01:18:49.879 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T01:18:49.879 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2023-07-06T01:18:49.879 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T01:18:49.880 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T01:18:49.880 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T01:18:49.880 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T01:18:50.326 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2023-07-06T01:18:50.328 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2023-07-06T01:18:50.328 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-07-06T01:18:50.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-07-06T01:18:50.529 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-07-06T01:18:50.598 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T01:18:50.598 INFO:tasks.ceph:role: osd.7 2023-07-06T01:18:50.599 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi103.front.sepia.ceph.com 2023-07-06T01:18:50.599 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-06T01:18:50.674 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T01:18:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T01:18:50.674 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T01:18:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T01:18:50.675 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T01:18:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2023-07-06T01:18:50.675 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T01:18:50.675 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T01:18:50.675 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T01:18:50.675 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T01:18:51.124 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2023-07-06T01:18:51.125 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2023-07-06T01:18:51.125 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-07-06T01:18:51.312 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-07-06T01:18:51.350 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:18:51.458 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:51.456+0000 7fc76b7f9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-07-06T01:18:51.458 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:51.456+0000 7fc76b7f9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-07-06T01:18:51.459 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:51.456+0000 7fc76b7f9640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T01:18:51.459 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:51.456+0000 7fc76b7f9640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T01:18:51.459 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:51.456+0000 7fc76b7f9640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T01:18:51.459 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:51.456+0000 7fc76b7f9640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-07-06T01:18:54.325 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-07-06T01:18:54.357 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:18:54.447 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:54.445+0000 7fa6f29f5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-07-06T01:18:54.447 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:54.445+0000 7fa6f29f5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-07-06T01:18:54.447 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:54.445+0000 7fa6f29f5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T01:18:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:54.445+0000 7fa6f29f5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T01:18:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:54.445+0000 7fa6f29f5640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T01:18:54.448 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:54.446+0000 7fa6f29f5640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-07-06T01:18:57.307 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-07-06T01:18:57.338 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:18:57.431 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:57.429+0000 7fa50a423640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-07-06T01:18:57.431 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:57.429+0000 7fa50a423640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-07-06T01:18:57.431 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:57.429+0000 7fa50a423640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T01:18:57.432 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:57.429+0000 7fa50a423640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T01:18:57.432 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:57.429+0000 7fa50a423640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T01:18:57.432 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:18:57.430+0000 7fa50a423640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-07-06T01:19:00.301 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-07-06T01:19:00.333 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:19:00.423 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:00.421+0000 7f2fe28dd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-07-06T01:19:00.424 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:00.422+0000 7f2fe28dd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-07-06T01:19:00.424 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:00.422+0000 7f2fe28dd640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T01:19:00.424 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:00.422+0000 7f2fe28dd640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T01:19:00.424 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:00.422+0000 7f2fe28dd640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T01:19:00.425 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:00.422+0000 7f2fe28dd640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-07-06T01:19:01.501 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:19:01.503 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:19:03.281 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-07-06T01:19:03.312 INFO:tasks.ceph:Reading keys from all nodes... 2023-07-06T01:19:03.312 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:03.313 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-07-06T01:19:03.345 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:03.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-07-06T01:19:03.417 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:03.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-07-06T01:19:03.487 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:03.488 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-07-06T01:19:03.557 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:03.557 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-07-06T01:19:03.625 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:03.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-07-06T01:19:03.694 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:03.694 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-07-06T01:19:03.761 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:03.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-07-06T01:19:03.793 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:03.794 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-07-06T01:19:03.865 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:03.866 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-07-06T01:19:03.937 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:03.937 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-07-06T01:19:04.006 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:04.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-07-06T01:19:04.075 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:04.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-07-06T01:19:04.145 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:04.145 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-07-06T01:19:04.212 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:04.213 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-07-06T01:19:04.234 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:04.234 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-07-06T01:19:04.294 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:04.294 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-07-06T01:19:04.312 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:04.312 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-07-06T01:19:04.370 INFO:tasks.ceph:Adding keys to all mons... 2023-07-06T01:19:04.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-06T01:19:04.372 DEBUG:teuthology.orchestra.run.smithi103:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-06T01:19:04.413 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr.y] 2023-07-06T01:19:04.414 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBqFqZkxDmKARAAFtK3jIDy4CK63pSxXdfYgA== 2023-07-06T01:19:04.414 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2023-07-06T01:19:04.414 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBqFqZkV6iICRAA1MCk3dMsqNkCf72IGijUcw== 2023-07-06T01:19:04.414 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2023-07-06T01:19:04.414 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBqFqZkAC7dEBAAwFbKmtddOUgk+zFNHeXARA== 2023-07-06T01:19:04.414 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.0] 2023-07-06T01:19:04.415 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBsFqZkm0BeExAAI1tQQ8r8sP59nJtjyvb0JQ== 2023-07-06T01:19:04.415 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.1] 2023-07-06T01:19:04.415 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBvFqZk7FLJEhAAnu8BLLGSpyL1ZF+YyXbRmA== 2023-07-06T01:19:04.415 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.2] 2023-07-06T01:19:04.415 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQByFqZkowUtFRAAVagTRRRKP99vm4ax6e5OzA== 2023-07-06T01:19:04.415 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.3] 2023-07-06T01:19:04.415 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQB1FqZkx0pyFRAAjYI2ThVYi4um9goY+b4hWQ== 2023-07-06T01:19:04.416 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr.x] 2023-07-06T01:19:04.416 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBqFqZkL8wRBhAAbGoxdH55UXARG50fMSNM9w== 2023-07-06T01:19:04.416 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2023-07-06T01:19:04.416 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBqFqZkA0TfFRAArj+u5qjLMmHyy1NJ09DUyQ== 2023-07-06T01:19:04.416 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2023-07-06T01:19:04.416 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBqFqZkWCkbHRAAjWrgjyK0BdXxv4ThUrFtYg== 2023-07-06T01:19:04.416 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.4] 2023-07-06T01:19:04.417 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQB7FqZka3s3GxAAypfzEphyjKJ1tHo9Yl9nMQ== 2023-07-06T01:19:04.417 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.5] 2023-07-06T01:19:04.417 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQB+FqZk1eOUGhAAnxp7kVV8jkttPyEFjrHVTQ== 2023-07-06T01:19:04.417 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.6] 2023-07-06T01:19:04.417 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQCBFqZk27ehGRAAROQLW6hs1IZY5GlwkdhwGA== 2023-07-06T01:19:04.418 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.7] 2023-07-06T01:19:04.418 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQCEFqZk4z0wGRAAWJ0U5aaSyQ8vOdbI0rk5+g== 2023-07-06T01:19:04.418 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2023-07-06T01:19:04.418 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBqFqZkwHh1IRAAx8BVLni20wF8yL0mJjooBQ== 2023-07-06T01:19:04.419 INFO:teuthology.orchestra.run.smithi022.stdout:[client.2] 2023-07-06T01:19:04.419 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBqFqZky+nZJBAABn14hveejk0SaouG1DVesw== 2023-07-06T01:19:04.419 INFO:teuthology.orchestra.run.smithi022.stdout:[client.1] 2023-07-06T01:19:04.419 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBqFqZkb9FaLBAATGB1K++aTCBst5UwjU16zw== 2023-07-06T01:19:04.420 INFO:teuthology.orchestra.run.smithi022.stdout:[client.3] 2023-07-06T01:19:04.420 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBqFqZkKlurLxAAgM2osFvBfK2mlvgsxyuPyQ== 2023-07-06T01:19:04.437 INFO:teuthology.orchestra.run.smithi103.stdout:[mgr.y] 2023-07-06T01:19:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBqFqZkxDmKARAAFtK3jIDy4CK63pSxXdfYgA== 2023-07-06T01:19:04.438 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.a] 2023-07-06T01:19:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBqFqZkV6iICRAA1MCk3dMsqNkCf72IGijUcw== 2023-07-06T01:19:04.438 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.c] 2023-07-06T01:19:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBqFqZkAC7dEBAAwFbKmtddOUgk+zFNHeXARA== 2023-07-06T01:19:04.438 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.0] 2023-07-06T01:19:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBsFqZkm0BeExAAI1tQQ8r8sP59nJtjyvb0JQ== 2023-07-06T01:19:04.439 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.1] 2023-07-06T01:19:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBvFqZk7FLJEhAAnu8BLLGSpyL1ZF+YyXbRmA== 2023-07-06T01:19:04.439 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.2] 2023-07-06T01:19:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQByFqZkowUtFRAAVagTRRRKP99vm4ax6e5OzA== 2023-07-06T01:19:04.439 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.3] 2023-07-06T01:19:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQB1FqZkx0pyFRAAjYI2ThVYi4um9goY+b4hWQ== 2023-07-06T01:19:04.439 INFO:teuthology.orchestra.run.smithi103.stdout:[mgr.x] 2023-07-06T01:19:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBqFqZkL8wRBhAAbGoxdH55UXARG50fMSNM9w== 2023-07-06T01:19:04.440 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.b] 2023-07-06T01:19:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBqFqZkA0TfFRAArj+u5qjLMmHyy1NJ09DUyQ== 2023-07-06T01:19:04.440 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.d] 2023-07-06T01:19:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBqFqZkWCkbHRAAjWrgjyK0BdXxv4ThUrFtYg== 2023-07-06T01:19:04.440 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.4] 2023-07-06T01:19:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQB7FqZka3s3GxAAypfzEphyjKJ1tHo9Yl9nMQ== 2023-07-06T01:19:04.440 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.5] 2023-07-06T01:19:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQB+FqZk1eOUGhAAnxp7kVV8jkttPyEFjrHVTQ== 2023-07-06T01:19:04.441 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.6] 2023-07-06T01:19:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQCBFqZk27ehGRAAROQLW6hs1IZY5GlwkdhwGA== 2023-07-06T01:19:04.441 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.7] 2023-07-06T01:19:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQCEFqZk4z0wGRAAWJ0U5aaSyQ8vOdbI0rk5+g== 2023-07-06T01:19:04.441 INFO:teuthology.orchestra.run.smithi103.stdout:[client.0] 2023-07-06T01:19:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBqFqZkwHh1IRAAx8BVLni20wF8yL0mJjooBQ== 2023-07-06T01:19:04.442 INFO:teuthology.orchestra.run.smithi103.stdout:[client.2] 2023-07-06T01:19:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBqFqZky+nZJBAABn14hveejk0SaouG1DVesw== 2023-07-06T01:19:04.442 INFO:teuthology.orchestra.run.smithi103.stdout:[client.1] 2023-07-06T01:19:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBqFqZkb9FaLBAATGB1K++aTCBst5UwjU16zw== 2023-07-06T01:19:04.442 INFO:teuthology.orchestra.run.smithi103.stdout:[client.3] 2023-07-06T01:19:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBqFqZkKlurLxAAgM2osFvBfK2mlvgsxyuPyQ== 2023-07-06T01:19:04.443 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T01:19:04.464 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T01:19:04.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T01:19:04.542 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T01:19:04.599 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T01:19:04.601 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T01:19:04.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:04.661 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:04.713 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:04.716 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:04.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:04.771 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:04.824 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:04.826 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:04.878 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T01:19:04.881 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T01:19:04.933 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T01:19:04.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T01:19:04.991 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T01:19:04.994 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T01:19:05.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:05.052 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:05.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:05.108 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:05.161 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:05.163 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:05.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:05.218 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T01:19:05.271 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T01:19:05.273 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T01:19:05.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T01:19:05.333 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T01:19:05.389 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T01:19:05.391 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T01:19:05.447 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T01:19:05.449 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T01:19:05.505 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-07-06T01:19:05.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-07-06T01:19:05.532 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T01:19:05.631 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-07-06T01:19:05.658 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-07-06T01:19:05.684 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T01:19:05.782 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-07-06T01:19:05.808 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-07-06T01:19:05.874 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T01:19:05.972 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-07-06T01:19:05.998 DEBUG:teuthology.orchestra.run.smithi022:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:19:06.001 DEBUG:teuthology.orchestra.run.smithi103:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-06T01:19:06.055 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-07-06T01:19:06.056 INFO:tasks.ceph.mon.a:Restarting daemon 2023-07-06T01:19:06.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-07-06T01:19:06.059 INFO:tasks.ceph.mon.a:Started 2023-07-06T01:19:06.059 INFO:tasks.ceph.mon.b:Restarting daemon 2023-07-06T01:19:06.059 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-07-06T01:19:06.098 INFO:tasks.ceph.mon.b:Started 2023-07-06T01:19:06.099 INFO:tasks.ceph.mon.c:Restarting daemon 2023-07-06T01:19:06.099 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-07-06T01:19:06.102 INFO:tasks.ceph.mon.c:Started 2023-07-06T01:19:06.102 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-07-06T01:19:06.102 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-07-06T01:19:06.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-07-06T01:19:06.105 INFO:tasks.ceph.mgr.y:Started 2023-07-06T01:19:06.105 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-07-06T01:19:06.106 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-07-06T01:19:06.107 INFO:tasks.ceph.mgr.x:Started 2023-07-06T01:19:06.108 INFO:tasks.ceph:Setting crush tunables to default 2023-07-06T01:19:06.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd crush tunables default 2023-07-06T01:19:06.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.235+0000 7faafe17c700 1 Processor -- start 2023-07-06T01:19:06.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.235+0000 7faafe17c700 1 -- start start 2023-07-06T01:19:06.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.235+0000 7faafe17c700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7faaf806a710 0x7faaf806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:06.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.235+0000 7faafe17c700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf80af670 con 0x7faaf80af2a0 2023-07-06T01:19:06.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.235+0000 7faafe17c700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf806d030 con 0x7faaf80cbc10 2023-07-06T01:19:06.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.235+0000 7faafe17c700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf80cc780 con 0x7faaf806a710 2023-07-06T01:19:06.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.236+0000 7faaf77fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7faaf80af2a0 0x7faaf806a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35428/0 (socket says 172.21.15.22:35428) 2023-07-06T01:19:06.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.236+0000 7faaf77fe700 1 -- 172.21.15.22:0/2503588157 learned_addr learned my addr 172.21.15.22:0/2503588157 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:06.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.236+0000 7faaf6ffd700 1 --2- 172.21.15.22:0/2503588157 >> v2:172.21.15.103:3300/0 conn(0x7faaf806a710 0x7faaf806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:06.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.236+0000 7faaf6ffd700 1 -- 172.21.15.22:0/2503588157 >> v2:172.21.15.103:3300/0 conn(0x7faaf806a710 msgr2=0x7faaf806caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-07-06T01:19:06.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.236+0000 7faaf6ffd700 1 -- 172.21.15.22:0/2503588157 >> v2:172.21.15.103:3300/0 conn(0x7faaf806a710 msgr2=0x7faaf806caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-06T01:19:06.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.236+0000 7faaf6ffd700 1 --2- 172.21.15.22:0/2503588157 >> v2:172.21.15.103:3300/0 conn(0x7faaf806a710 0x7faaf806caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-06T01:19:06.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.236+0000 7faaf6ffd700 1 --2- 172.21.15.22:0/2503588157 >> v2:172.21.15.103:3300/0 conn(0x7faaf806a710 0x7faaf806caf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T01:19:06.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.419+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023137389 0 0) 0x7faaf80af670 con 0x7faaf80af2a0 2023-07-06T01:19:06.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.419+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faae00035b0 con 0x7faaf80af2a0 2023-07-06T01:19:06.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4095234391 0 0) 0x7faae00035b0 con 0x7faaf80af2a0 2023-07-06T01:19:06.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 >> v2:172.21.15.103:3300/0 conn(0x7faaf806a710 msgr2=0x7faaf806caf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T01:19:06.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf67fc700 1 --2- 172.21.15.22:0/2503588157 >> v2:172.21.15.103:3300/0 conn(0x7faaf806a710 0x7faaf806caf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:06.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 >> v1:172.21.15.103:6789/0 conn(0x7faaf80cbc10 legacy=0x7faaf80cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:06.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf80cd560 con 0x7faaf80af2a0 2023-07-06T01:19:06.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7faae8001670 con 0x7faaf80af2a0 2023-07-06T01:19:06.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 >> v1:172.21.15.22:6789/0 conn(0x7faaf80af2a0 legacy=0x7faaf806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:06.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf67fc700 1 --2- 172.21.15.22:0/2503588157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae0003a50 0x7faae0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:06.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf67fc700 1 --2- 172.21.15.22:0/2503588157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0004850 0x7faae0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:06.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf67fc700 1 --2- 172.21.15.22:0/2503588157 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faae0005250 0x7faae000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:06.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf80cc780 con 0x7faae0003a50 2023-07-06T01:19:06.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae000c190 con 0x7faae0004850 2023-07-06T01:19:06.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae000c350 con 0x7faae0005250 2023-07-06T01:19:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf77fe700 1 --2- 172.21.15.22:0/2503588157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae0003a50 0x7faae0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf7fff700 1 --2- 172.21.15.22:0/2503588157 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faae0005250 0x7faae000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf6ffd700 1 --2- 172.21.15.22:0/2503588157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0004850 0x7faae0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf77fe700 1 -- 172.21.15.22:0/2503588157 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faae0005250 msgr2=0x7faae000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf77fe700 1 --2- 172.21.15.22:0/2503588157 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faae0005250 0x7faae000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.420+0000 7faaf77fe700 1 -- 172.21.15.22:0/2503588157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0004850 msgr2=0x7faae0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:06.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.421+0000 7faaf77fe700 1 --2- 172.21.15.22:0/2503588157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0004850 0x7faae0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.421+0000 7faaf7fff700 1 -- 172.21.15.22:0/2503588157 reap_dead start 2023-07-06T01:19:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.421+0000 7faaf77fe700 1 -- 172.21.15.22:0/2503588157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae00035b0 con 0x7faae0003a50 2023-07-06T01:19:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.421+0000 7faaf6ffd700 1 --2- 172.21.15.22:0/2503588157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faae0004850 0x7faae0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.421+0000 7faaf77fe700 1 --2- 172.21.15.22:0/2503588157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae0003a50 0x7faae0003e60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faae0005830 tx=0x7faae0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cccda3286e2d00 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.421+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8007b70 con 0x7faae0003a50 2023-07-06T01:19:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.421+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae800cca0 con 0x7faae0003a50 2023-07-06T01:19:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.421+0000 7faaf67fc700 1 -- 172.21.15.22:0/2503588157 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8005ba0 con 0x7faae0003a50 2023-07-06T01:19:06.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.422+0000 7faafe17c700 1 -- 172.21.15.22:0/2503588157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae0003a50 msgr2=0x7faae0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:06.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.422+0000 7faafe17c700 1 --2- 172.21.15.22:0/2503588157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae0003a50 0x7faae0003e60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faae0005830 tx=0x7faae0005880 comp rx=0 tx=0).stop 2023-07-06T01:19:06.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.422+0000 7faafe17c700 1 -- 172.21.15.22:0/2503588157 shutdown_connections 2023-07-06T01:19:06.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.422+0000 7faafe17c700 1 --2- 172.21.15.22:0/2503588157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae0003a50 0x7faae0003e60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:06.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.422+0000 7faafe17c700 1 -- 172.21.15.22:0/2503588157 >> 172.21.15.22:0/2503588157 conn(0x7faaf80b1890 msgr2=0x7faaf80b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:06.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.422+0000 7faafe17c700 1 -- 172.21.15.22:0/2503588157 shutdown_connections 2023-07-06T01:19:06.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.422+0000 7faafe17c700 1 -- 172.21.15.22:0/2503588157 wait complete. 2023-07-06T01:19:06.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.423+0000 7faafe17c700 1 Processor -- start 2023-07-06T01:19:06.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.423+0000 7faafe17c700 1 -- start start 2023-07-06T01:19:06.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faafe17c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae0003a50 0x7faaf81556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:06.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faafe17c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faaf8155e30 0x7faaf815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:06.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faafe17c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faaf8156380 0x7faaf80c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:06.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faafe17c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf8155c20 con 0x7faae0003a50 2023-07-06T01:19:06.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faafe17c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf815d030 con 0x7faaf8156380 2023-07-06T01:19:06.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faafe17c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf815d310 con 0x7faaf8155e30 2023-07-06T01:19:06.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faaf77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae0003a50 0x7faaf81556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:06.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faaf77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae0003a50 0x7faaf81556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44710/0 (socket says 172.21.15.22:44710) 2023-07-06T01:19:06.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faaf77fe700 1 -- 172.21.15.22:0/3082025427 learned_addr learned my addr 172.21.15.22:0/3082025427 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:06.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faaf7fff700 1 --2- 172.21.15.22:0/3082025427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faaf8156380 0x7faaf80c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:06.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faaf6ffd700 1 --2- 172.21.15.22:0/3082025427 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faaf8155e30 0x7faaf815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:06.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faaf77fe700 1 -- 172.21.15.22:0/3082025427 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faaf8155e30 msgr2=0x7faaf815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:06.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faaf77fe700 1 --2- 172.21.15.22:0/3082025427 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faaf8155e30 0x7faaf815e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:06.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faaf77fe700 1 -- 172.21.15.22:0/3082025427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faaf8156380 msgr2=0x7faaf80c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:06.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.424+0000 7faaf77fe700 1 --2- 172.21.15.22:0/3082025427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faaf8156380 0x7faaf80c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:06.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.425+0000 7faaf77fe700 1 -- 172.21.15.22:0/3082025427 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf815d5f0 con 0x7faae0003a50 2023-07-06T01:19:06.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.425+0000 7faaf77fe700 1 --2- 172.21.15.22:0/3082025427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae0003a50 0x7faaf81556e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faae000c160 tx=0x7faae8001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:06.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.426+0000 7faadffff700 1 -- 172.21.15.22:0/3082025427 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8001430 con 0x7faae0003a50 2023-07-06T01:19:06.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.426+0000 7faadffff700 1 -- 172.21.15.22:0/3082025427 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae8001590 con 0x7faae0003a50 2023-07-06T01:19:06.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.426+0000 7faadffff700 1 -- 172.21.15.22:0/3082025427 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8007310 con 0x7faae0003a50 2023-07-06T01:19:06.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.426+0000 7faafe17c700 1 -- 172.21.15.22:0/3082025427 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf815d8d0 con 0x7faae0003a50 2023-07-06T01:19:06.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.426+0000 7faafe17c700 1 -- 172.21.15.22:0/3082025427 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf815ccc0 con 0x7faae0003a50 2023-07-06T01:19:06.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.426+0000 7faadffff700 1 -- 172.21.15.22:0/3082025427 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faae8027450 con 0x7faae0003a50 2023-07-06T01:19:06.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.426+0000 7faadffff700 1 -- 172.21.15.22:0/3082025427 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faae801e770 con 0x7faae0003a50 2023-07-06T01:19:06.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.427+0000 7faafe17c700 1 -- 172.21.15.22:0/3082025427 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf815ddf0 con 0x7faae0003a50 2023-07-06T01:19:06.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.429+0000 7faadffff700 1 -- 172.21.15.22:0/3082025427 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7faaf815ddf0 con 0x7faae0003a50 2023-07-06T01:19:06.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.507+0000 7faafe17c700 1 -- 172.21.15.22:0/3082025427 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7faaf80744b0 con 0x7faae0003a50 2023-07-06T01:19:06.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.509+0000 7faadffff700 1 -- 172.21.15.22:0/3082025427 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7faae8022070 con 0x7faae0003a50 2023-07-06T01:19:06.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.510+0000 7faafe17c700 1 -- 172.21.15.22:0/3082025427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae0003a50 msgr2=0x7faaf81556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:06.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.510+0000 7faafe17c700 1 --2- 172.21.15.22:0/3082025427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae0003a50 0x7faaf81556e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faae000c160 tx=0x7faae8001930 comp rx=0 tx=0).stop 2023-07-06T01:19:06.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.510+0000 7faafe17c700 1 -- 172.21.15.22:0/3082025427 shutdown_connections 2023-07-06T01:19:06.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.510+0000 7faafe17c700 1 --2- 172.21.15.22:0/3082025427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faaf8156380 0x7faaf80c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:06.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.510+0000 7faafe17c700 1 --2- 172.21.15.22:0/3082025427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faae0003a50 0x7faaf81556e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:06.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.510+0000 7faafe17c700 1 --2- 172.21.15.22:0/3082025427 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faaf8155e30 0x7faaf815e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:06.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.510+0000 7faafe17c700 1 -- 172.21.15.22:0/3082025427 >> 172.21.15.22:0/3082025427 conn(0x7faaf80b1890 msgr2=0x7faaf80ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:06.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.510+0000 7faafe17c700 1 -- 172.21.15.22:0/3082025427 shutdown_connections 2023-07-06T01:19:06.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:06.510+0000 7faafe17c700 1 -- 172.21.15.22:0/3082025427 wait complete. 2023-07-06T01:19:06.514 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2023-07-06T01:19:06.524 INFO:tasks.ceph:check_enable_crimson: False 2023-07-06T01:19:06.525 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-07-06T01:19:06.525 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:06.526 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-07-06T01:19:06.540 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:06.539+0000 7f786f557200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T01:19:06.547 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:06.546+0000 7fde73547200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T01:19:06.553 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:06.553 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-07-06T01:19:06.606 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:06.604+0000 7f786f557200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T01:19:06.618 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:06.616+0000 7fde73547200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T01:19:06.620 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:06.621 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-07-06T01:19:06.680 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:06.679+0000 7f786f557200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T01:19:06.688 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:06.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-07-06T01:19:06.700 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:06.698+0000 7fde73547200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T01:19:06.727 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:06.728 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-07-06T01:19:06.757 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:06.757 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-07-06T01:19:06.760 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:06.758+0000 7f786f557200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T01:19:06.785 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:06.783+0000 7fde73547200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T01:19:06.786 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:06.786 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-07-06T01:19:06.852 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:06.854 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-07-06T01:19:06.909 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:06.907+0000 7f786f557200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T01:19:06.920 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new a2ddc168-d6c5-4312-b30d-71c083434002 0 2023-07-06T01:19:06.934 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:06.932+0000 7fde73547200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T01:19:07.080 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.078+0000 7fcb167ac700 1 Processor -- start 2023-07-06T01:19:07.081 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.078+0000 7fcb167ac700 1 -- start start 2023-07-06T01:19:07.083 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.079+0000 7fcb167ac700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fcb1015df50 0x7fcb1006cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.083 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.079+0000 7fcb167ac700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcb100cbc30 0x7fcb10070d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.083 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.079+0000 7fcb167ac700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb100c9370 con 0x7fcb100b1a10 2023-07-06T01:19:07.083 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.079+0000 7fcb167ac700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb100c9650 con 0x7fcb100cbc30 2023-07-06T01:19:07.083 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.079+0000 7fcb167ac700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb100c9930 con 0x7fcb1015df50 2023-07-06T01:19:07.083 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.079+0000 7fcb0ffff700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fcb1015df50 0x7fcb1006cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.084 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.079+0000 7fcb0f7fe700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fcb100b1a10 0x7fcb100cb520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:52318/0 (socket says 172.21.15.103:52318) 2023-07-06T01:19:07.084 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.079+0000 7fcb0f7fe700 1 -- 172.21.15.103:0/4139259233 learned_addr learned my addr 172.21.15.103:0/4139259233 (peer_addr_for_me v1:172.21.15.103:0/0) 2023-07-06T01:19:07.084 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.079+0000 7fcb14d49700 1 --2- 172.21.15.103:0/4139259233 >> v2:172.21.15.22:3300/0 conn(0x7fcb100cbc30 0x7fcb10070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.084 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb0ffff700 1 -- 172.21.15.103:0/4139259233 >> v2:172.21.15.22:3300/0 conn(0x7fcb100cbc30 msgr2=0x7fcb10070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:07.084 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb0ffff700 1 --2- 172.21.15.103:0/4139259233 >> v2:172.21.15.22:3300/0 conn(0x7fcb100cbc30 0x7fcb10070d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.084 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb0ffff700 1 -- 172.21.15.103:0/4139259233 >> v1:172.21.15.103:6789/0 conn(0x7fcb100b1a10 legacy=0x7fcb100cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.084 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb0effd700 1 -- 172.21.15.103:0/4139259233 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750019323 0 0) 0x7fcb100c9370 con 0x7fcb100b1a10 2023-07-06T01:19:07.085 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb0ffff700 1 -- 172.21.15.103:0/4139259233 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb100c9c10 con 0x7fcb1015df50 2023-07-06T01:19:07.085 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb0ffff700 1 --2- 172.21.15.103:0/4139259233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb1015df50 0x7fcb1006cec0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcb100cc700 tx=0x7fcb00009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89b178011db78c12 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:07.085 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb0effd700 1 -- 172.21.15.103:0/4139259233 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00011830 con 0x7fcb1015df50 2023-07-06T01:19:07.085 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb0effd700 1 -- 172.21.15.103:0/4139259233 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb00011990 con 0x7fcb1015df50 2023-07-06T01:19:07.085 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb0effd700 1 -- 172.21.15.103:0/4139259233 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00011d20 con 0x7fcb1015df50 2023-07-06T01:19:07.085 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb167ac700 1 -- 172.21.15.103:0/4139259233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb1015df50 msgr2=0x7fcb1006cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.085 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb167ac700 1 --2- 172.21.15.103:0/4139259233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb1015df50 0x7fcb1006cec0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcb100cc700 tx=0x7fcb00009b20 comp rx=0 tx=0).stop 2023-07-06T01:19:07.085 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb167ac700 1 -- 172.21.15.103:0/4139259233 shutdown_connections 2023-07-06T01:19:07.086 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb167ac700 1 --2- 172.21.15.103:0/4139259233 >> v2:172.21.15.22:3300/0 conn(0x7fcb100cbc30 0x7fcb10070d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.086 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb167ac700 1 --2- 172.21.15.103:0/4139259233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb1015df50 0x7fcb1006cec0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.086 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb167ac700 1 -- 172.21.15.103:0/4139259233 >> 172.21.15.103:0/4139259233 conn(0x7fcb100b0710 msgr2=0x7fcb100b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:07.086 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb167ac700 1 -- 172.21.15.103:0/4139259233 shutdown_connections 2023-07-06T01:19:07.086 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.080+0000 7fcb167ac700 1 -- 172.21.15.103:0/4139259233 wait complete. 2023-07-06T01:19:07.088 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.082+0000 7fcb167ac700 1 Processor -- start 2023-07-06T01:19:07.088 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.082+0000 7fcb167ac700 1 -- start start 2023-07-06T01:19:07.088 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb167ac700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb100b1a10 0x7fcb10152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.088 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb167ac700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb100cbc30 0x7fcb100c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.089 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb167ac700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcb1015df50 0x7fcb100c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.089 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb167ac700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb100cd190 con 0x7fcb100b1a10 2023-07-06T01:19:07.089 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb167ac700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb10153350 con 0x7fcb100cbc30 2023-07-06T01:19:07.089 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb167ac700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb100cc7f0 con 0x7fcb1015df50 2023-07-06T01:19:07.089 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb100cbc30 0x7fcb100c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.089 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0f7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb100cbc30 0x7fcb100c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47486/0 (socket says 172.21.15.103:47486) 2023-07-06T01:19:07.089 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0f7fe700 1 -- 172.21.15.103:0/2861451624 learned_addr learned my addr 172.21.15.103:0/2861451624 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-07-06T01:19:07.089 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb14d49700 1 --2- 172.21.15.103:0/2861451624 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcb1015df50 0x7fcb100c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0ffff700 1 --2- 172.21.15.103:0/2861451624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb100b1a10 0x7fcb10152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0f7fe700 1 -- 172.21.15.103:0/2861451624 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcb1015df50 msgr2=0x7fcb100c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0f7fe700 1 --2- 172.21.15.103:0/2861451624 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcb1015df50 0x7fcb100c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0f7fe700 1 -- 172.21.15.103:0/2861451624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb100b1a10 msgr2=0x7fcb10152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0f7fe700 1 --2- 172.21.15.103:0/2861451624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb100b1a10 0x7fcb10152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0f7fe700 1 -- 172.21.15.103:0/2861451624 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb000094a0 con 0x7fcb100cbc30 2023-07-06T01:19:07.090 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0ffff700 1 --2- 172.21.15.103:0/2861451624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb100b1a10 0x7fcb10152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0f7fe700 1 --2- 172.21.15.103:0/2861451624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb100cbc30 0x7fcb100c3f20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400c540 tx=0x7fcb0400c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0cff9700 1 -- 172.21.15.103:0/2861451624 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04004c80 con 0x7fcb100cbc30 2023-07-06T01:19:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0cff9700 1 -- 172.21.15.103:0/2861451624 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb04004de0 con 0x7fcb100cbc30 2023-07-06T01:19:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb167ac700 1 -- 172.21.15.103:0/2861451624 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb100ccad0 con 0x7fcb100cbc30 2023-07-06T01:19:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb0cff9700 1 -- 172.21.15.103:0/2861451624 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04007310 con 0x7fcb100cbc30 2023-07-06T01:19:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.083+0000 7fcb167ac700 1 -- 172.21.15.103:0/2861451624 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb100ae120 con 0x7fcb100cbc30 2023-07-06T01:19:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.084+0000 7fcb0cff9700 1 -- 172.21.15.103:0/2861451624 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcb0401d3f0 con 0x7fcb100cbc30 2023-07-06T01:19:07.091 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.084+0000 7fcb0cff9700 1 -- 172.21.15.103:0/2861451624 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcb04013700 con 0x7fcb100cbc30 2023-07-06T01:19:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.084+0000 7fcb167ac700 1 -- 172.21.15.103:0/2861451624 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb100744b0 con 0x7fcb100cbc30 2023-07-06T01:19:07.092 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.086+0000 7fcb0cff9700 1 -- 172.21.15.103:0/2861451624 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fcb0401d8d0 con 0x7fcb100cbc30 2023-07-06T01:19:07.171 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.169+0000 7fcb167ac700 1 -- 172.21.15.103:0/2861451624 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a2ddc168-d6c5-4312-b30d-71c083434002", "id": 0} v 0) v1 -- 0x7fcb100cb300 con 0x7fcb100cbc30 2023-07-06T01:19:07.176 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.174+0000 7fcb0cff9700 1 -- 172.21.15.103:0/2861451624 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a2ddc168-d6c5-4312-b30d-71c083434002", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcb0401dba0 con 0x7fcb100cbc30 2023-07-06T01:19:07.176 INFO:teuthology.orchestra.run.smithi103.stdout:0 2023-07-06T01:19:07.177 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.175+0000 7fcb167ac700 1 -- 172.21.15.103:0/2861451624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb100cbc30 msgr2=0x7fcb100c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.177 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.175+0000 7fcb167ac700 1 --2- 172.21.15.103:0/2861451624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb100cbc30 0x7fcb100c3f20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400c540 tx=0x7fcb0400c850 comp rx=0 tx=0).stop 2023-07-06T01:19:07.178 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.175+0000 7fcb167ac700 1 -- 172.21.15.103:0/2861451624 shutdown_connections 2023-07-06T01:19:07.178 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.175+0000 7fcb167ac700 1 --2- 172.21.15.103:0/2861451624 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcb1015df50 0x7fcb100c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.178 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.175+0000 7fcb167ac700 1 --2- 172.21.15.103:0/2861451624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb100b1a10 0x7fcb10152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.178 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.175+0000 7fcb167ac700 1 --2- 172.21.15.103:0/2861451624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb100cbc30 0x7fcb100c3f20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.178 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.175+0000 7fcb167ac700 1 -- 172.21.15.103:0/2861451624 >> 172.21.15.103:0/2861451624 conn(0x7fcb100b0710 msgr2=0x7fcb10070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:07.178 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.175+0000 7fcb167ac700 1 -- 172.21.15.103:0/2861451624 shutdown_connections 2023-07-06T01:19:07.178 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.175+0000 7fcb167ac700 1 -- 172.21.15.103:0/2861451624 wait complete. 2023-07-06T01:19:07.189 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new f25f28cf-f149-4a01-b035-e9104cb10927 1 2023-07-06T01:19:07.259 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:07.257+0000 7f786f557200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T01:19:07.278 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:07.276+0000 7fde73547200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T01:19:07.315 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.313+0000 7f30b3961700 1 Processor -- start 2023-07-06T01:19:07.315 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.314+0000 7f30b3961700 1 -- start start 2023-07-06T01:19:07.318 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.314+0000 7f30b3961700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30ac0ccd70 0x7f30ac15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.318 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.314+0000 7f30b3961700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f30ac06d040 0x7f30ac06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.319 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.314+0000 7f30b3961700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ac0cf230 con 0x7f30ac159360 2023-07-06T01:19:07.319 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.314+0000 7f30b3961700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ac15df20 con 0x7f30ac0ccd70 2023-07-06T01:19:07.319 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.314+0000 7f30b3961700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ac0ce620 con 0x7f30ac06d040 2023-07-06T01:19:07.319 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.315+0000 7f30b1efe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f30ac06d040 0x7f30ac06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.320 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.315+0000 7f30b16fd700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f30ac159360 0x7f30ac0cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:52328/0 (socket says 172.21.15.103:52328) 2023-07-06T01:19:07.320 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.315+0000 7f30b16fd700 1 -- 172.21.15.103:0/1534393291 learned_addr learned my addr 172.21.15.103:0/1534393291 (peer_addr_for_me v1:172.21.15.103:0/0) 2023-07-06T01:19:07.320 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.315+0000 7f30b0efc700 1 --2- 172.21.15.103:0/1534393291 >> v2:172.21.15.22:3300/0 conn(0x7f30ac0ccd70 0x7f30ac15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.320 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.315+0000 7f30b1efe700 1 -- 172.21.15.103:0/1534393291 >> v2:172.21.15.22:3300/0 conn(0x7f30ac0ccd70 msgr2=0x7f30ac15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:07.320 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.315+0000 7f30b1efe700 1 --2- 172.21.15.103:0/1534393291 >> v2:172.21.15.22:3300/0 conn(0x7f30ac0ccd70 0x7f30ac15d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.321 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.315+0000 7f30b1efe700 1 -- 172.21.15.103:0/1534393291 >> v1:172.21.15.103:6789/0 conn(0x7f30ac159360 legacy=0x7f30ac0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.321 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.315+0000 7f30b1efe700 1 -- 172.21.15.103:0/1534393291 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac0ce900 con 0x7f30ac06d040 2023-07-06T01:19:07.321 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.315+0000 7f30a3fff700 1 -- 172.21.15.103:0/1534393291 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138152464 0 0) 0x7f30ac0cf230 con 0x7f30ac159360 2023-07-06T01:19:07.321 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.315+0000 7f30b1efe700 1 --2- 172.21.15.103:0/1534393291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ac06d040 0x7f30ac06d4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0cd920 tx=0x7f30a400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d80e2e57b5f73f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:07.321 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.315+0000 7f30a3fff700 1 -- 172.21.15.103:0/1534393291 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a400e070 con 0x7f30ac06d040 2023-07-06T01:19:07.321 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.315+0000 7f30a3fff700 1 -- 172.21.15.103:0/1534393291 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a400bb20 con 0x7f30ac06d040 2023-07-06T01:19:07.321 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.315+0000 7f30a3fff700 1 -- 172.21.15.103:0/1534393291 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4012430 con 0x7f30ac06d040 2023-07-06T01:19:07.322 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.316+0000 7f30b3961700 1 -- 172.21.15.103:0/1534393291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ac06d040 msgr2=0x7f30ac06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.322 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.316+0000 7f30b3961700 1 --2- 172.21.15.103:0/1534393291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ac06d040 0x7f30ac06d4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0cd920 tx=0x7f30a400b330 comp rx=0 tx=0).stop 2023-07-06T01:19:07.322 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.316+0000 7f30b3961700 1 -- 172.21.15.103:0/1534393291 shutdown_connections 2023-07-06T01:19:07.322 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.316+0000 7f30b3961700 1 --2- 172.21.15.103:0/1534393291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ac06d040 0x7f30ac06d4a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.322 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.316+0000 7f30b3961700 1 --2- 172.21.15.103:0/1534393291 >> v2:172.21.15.22:3300/0 conn(0x7f30ac0ccd70 0x7f30ac15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.322 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.316+0000 7f30b3961700 1 -- 172.21.15.103:0/1534393291 >> 172.21.15.103:0/1534393291 conn(0x7f30ac0b1890 msgr2=0x7f30ac0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:07.322 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.316+0000 7f30b3961700 1 -- 172.21.15.103:0/1534393291 shutdown_connections 2023-07-06T01:19:07.322 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.316+0000 7f30b3961700 1 -- 172.21.15.103:0/1534393291 wait complete. 2023-07-06T01:19:07.324 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.317+0000 7f30b3961700 1 Processor -- start 2023-07-06T01:19:07.324 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.317+0000 7f30b3961700 1 -- start start 2023-07-06T01:19:07.325 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b3961700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ac06d040 0x7f30ac159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.325 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b3961700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ac0ccd70 0x7f30ac0b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.325 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b3961700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ac159360 0x7f30ac15bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.325 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b3961700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac154cf0 con 0x7f30ac159360 2023-07-06T01:19:07.325 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b3961700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac155700 con 0x7f30ac06d040 2023-07-06T01:19:07.325 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b3961700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac0b15b0 con 0x7f30ac0ccd70 2023-07-06T01:19:07.325 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b16fd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ac06d040 0x7f30ac159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.326 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b0efc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ac0ccd70 0x7f30ac0b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.326 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b16fd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ac06d040 0x7f30ac159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47496/0 (socket says 172.21.15.103:47496) 2023-07-06T01:19:07.326 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b16fd700 1 -- 172.21.15.103:0/2772606115 learned_addr learned my addr 172.21.15.103:0/2772606115 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-07-06T01:19:07.326 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b1efe700 1 --2- 172.21.15.103:0/2772606115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ac159360 0x7f30ac15bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.326 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b16fd700 1 -- 172.21.15.103:0/2772606115 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ac0ccd70 msgr2=0x7f30ac0b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.326 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b16fd700 1 --2- 172.21.15.103:0/2772606115 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ac0ccd70 0x7f30ac0b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.326 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b16fd700 1 -- 172.21.15.103:0/2772606115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ac159360 msgr2=0x7f30ac15bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.327 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b16fd700 1 --2- 172.21.15.103:0/2772606115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ac159360 0x7f30ac15bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.327 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b16fd700 1 -- 172.21.15.103:0/2772606115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a4009a70 con 0x7f30ac06d040 2023-07-06T01:19:07.327 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b0efc700 1 --2- 172.21.15.103:0/2772606115 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ac0ccd70 0x7f30ac0b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:07.327 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30b16fd700 1 --2- 172.21.15.103:0/2772606115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ac06d040 0x7f30ac159730 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f309c00c550 tx=0x7f309c00c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:07.327 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.318+0000 7f30a1ffb700 1 -- 172.21.15.103:0/2772606115 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c004c80 con 0x7f30ac06d040 2023-07-06T01:19:07.327 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.319+0000 7f30a1ffb700 1 -- 172.21.15.103:0/2772606115 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c004e00 con 0x7f30ac06d040 2023-07-06T01:19:07.327 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.319+0000 7f30b3961700 1 -- 172.21.15.103:0/2772606115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ac159e30 con 0x7f30ac06d040 2023-07-06T01:19:07.327 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.319+0000 7f30a1ffb700 1 -- 172.21.15.103:0/2772606115 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c007310 con 0x7f30ac06d040 2023-07-06T01:19:07.328 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.319+0000 7f30a1ffb700 1 -- 172.21.15.103:0/2772606115 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f309c0133c0 con 0x7f30ac06d040 2023-07-06T01:19:07.328 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.319+0000 7f30b3961700 1 -- 172.21.15.103:0/2772606115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ac15a3f0 con 0x7f30ac06d040 2023-07-06T01:19:07.328 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.319+0000 7f30a1ffb700 1 -- 172.21.15.103:0/2772606115 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f309c013c00 con 0x7f30ac06d040 2023-07-06T01:19:07.328 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.320+0000 7f30b3961700 1 -- 172.21.15.103:0/2772606115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ac0cf6a0 con 0x7f30ac06d040 2023-07-06T01:19:07.328 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.322+0000 7f30a1ffb700 1 -- 172.21.15.103:0/2772606115 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f30ac0cf6a0 con 0x7f30ac06d040 2023-07-06T01:19:07.338 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:07.337+0000 7f786f557200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T01:19:07.356 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:07.355+0000 7fde73547200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T01:19:07.408 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.406+0000 7f30b3961700 1 -- 172.21.15.103:0/2772606115 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f25f28cf-f149-4a01-b035-e9104cb10927", "id": 1} v 0) v1 -- 0x7f30ac0cb8e0 con 0x7f30ac06d040 2023-07-06T01:19:07.415 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.414+0000 7f30a1ffb700 1 -- 172.21.15.103:0/2772606115 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f25f28cf-f149-4a01-b035-e9104cb10927", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f309c014420 con 0x7f30ac06d040 2023-07-06T01:19:07.416 INFO:teuthology.orchestra.run.smithi103.stdout:1 2023-07-06T01:19:07.417 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.415+0000 7f30b3961700 1 -- 172.21.15.103:0/2772606115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ac06d040 msgr2=0x7f30ac159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.417 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.415+0000 7f30b3961700 1 --2- 172.21.15.103:0/2772606115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ac06d040 0x7f30ac159730 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f309c00c550 tx=0x7f309c00c860 comp rx=0 tx=0).stop 2023-07-06T01:19:07.417 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.415+0000 7f30b3961700 1 -- 172.21.15.103:0/2772606115 shutdown_connections 2023-07-06T01:19:07.417 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.415+0000 7f30b3961700 1 --2- 172.21.15.103:0/2772606115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ac159360 0x7f30ac15bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.417 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.415+0000 7f30b3961700 1 --2- 172.21.15.103:0/2772606115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ac06d040 0x7f30ac159730 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.417 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.415+0000 7f30b3961700 1 --2- 172.21.15.103:0/2772606115 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ac0ccd70 0x7f30ac0b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.418 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.415+0000 7f30b3961700 1 -- 172.21.15.103:0/2772606115 >> 172.21.15.103:0/2772606115 conn(0x7f30ac0b1890 msgr2=0x7f30ac06e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:07.418 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.415+0000 7f30b3961700 1 -- 172.21.15.103:0/2772606115 shutdown_connections 2023-07-06T01:19:07.418 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.415+0000 7f30b3961700 1 -- 172.21.15.103:0/2772606115 wait complete. 2023-07-06T01:19:07.430 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new a447ebab-c06d-4d60-a53c-e6847032aeba 2 2023-07-06T01:19:07.477 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:07.475+0000 7f786f557200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T01:19:07.551 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.549+0000 7f2b85c7f700 1 Processor -- start 2023-07-06T01:19:07.551 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.549+0000 7f2b85c7f700 1 -- start start 2023-07-06T01:19:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.550+0000 7f2b85c7f700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f2b8015c750 0x7f2b800cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.550+0000 7f2b85c7f700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2b800cdf80 0x7f2b8006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.550+0000 7f2b85c7f700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b800d06f0 con 0x7f2b800ce400 2023-07-06T01:19:07.553 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.550+0000 7f2b85c7f700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b800ceb90 con 0x7f2b800cdf80 2023-07-06T01:19:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.550+0000 7f2b85c7f700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b800cee70 con 0x7f2b8015c750 2023-07-06T01:19:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.550+0000 7f2b7f7fe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f2b8015c750 0x7f2b800cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.550+0000 7f2b7f7fe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f2b8015c750 0x7f2b800cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47502/0 (socket says 172.21.15.103:47502) 2023-07-06T01:19:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.550+0000 7f2b7f7fe700 1 -- 172.21.15.103:0/1868446257 learned_addr learned my addr 172.21.15.103:0/1868446257 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-07-06T01:19:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.550+0000 7f2b7f7fe700 1 -- 172.21.15.103:0/1868446257 >> v2:172.21.15.22:3300/0 conn(0x7f2b800cdf80 msgr2=0x7f2b8006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.550+0000 7f2b7f7fe700 1 --2- 172.21.15.103:0/1868446257 >> v2:172.21.15.22:3300/0 conn(0x7f2b800cdf80 0x7f2b8006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.554 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.550+0000 7f2b7f7fe700 1 -- 172.21.15.103:0/1868446257 >> v1:172.21.15.22:6789/0 conn(0x7f2b800ce400 legacy=0x7f2b800cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.550+0000 7f2b7f7fe700 1 -- 172.21.15.103:0/1868446257 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b800cf150 con 0x7f2b8015c750 2023-07-06T01:19:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.551+0000 7f2b7f7fe700 1 --2- 172.21.15.103:0/1868446257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b8015c750 0x7f2b800cfc30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2b80070f00 tx=0x7f2b7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1fe518ab740c903 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.551+0000 7f2b7e7fc700 1 -- 172.21.15.103:0/1868446257 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7000e070 con 0x7f2b8015c750 2023-07-06T01:19:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.551+0000 7f2b7e7fc700 1 -- 172.21.15.103:0/1868446257 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b70003a20 con 0x7f2b8015c750 2023-07-06T01:19:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.551+0000 7f2b7e7fc700 1 -- 172.21.15.103:0/1868446257 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70003dd0 con 0x7f2b8015c750 2023-07-06T01:19:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.551+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1868446257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b8015c750 msgr2=0x7f2b800cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.551+0000 7f2b85c7f700 1 --2- 172.21.15.103:0/1868446257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b8015c750 0x7f2b800cfc30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2b80070f00 tx=0x7f2b7000b330 comp rx=0 tx=0).stop 2023-07-06T01:19:07.555 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.551+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1868446257 shutdown_connections 2023-07-06T01:19:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.551+0000 7f2b85c7f700 1 --2- 172.21.15.103:0/1868446257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b8015c750 0x7f2b800cfc30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.551+0000 7f2b85c7f700 1 --2- 172.21.15.103:0/1868446257 >> v2:172.21.15.22:3300/0 conn(0x7f2b800cdf80 0x7f2b8006b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.551+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1868446257 >> 172.21.15.103:0/1868446257 conn(0x7f2b800999f0 msgr2=0x7f2b80099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.551+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1868446257 shutdown_connections 2023-07-06T01:19:07.556 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.551+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1868446257 wait complete. 2023-07-06T01:19:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.552+0000 7f2b85c7f700 1 Processor -- start 2023-07-06T01:19:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.553+0000 7f2b85c7f700 1 -- start start 2023-07-06T01:19:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.553+0000 7f2b85c7f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b800cdf80 0x7f2b80151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.558 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.553+0000 7f2b85c7f700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b800ce400 0x7f2b8015d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.553+0000 7f2b85c7f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b8015c750 0x7f2b8009aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.553+0000 7f2b85c7f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8009caa0 con 0x7f2b800cdf80 2023-07-06T01:19:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.553+0000 7f2b85c7f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b80151540 con 0x7f2b8015c750 2023-07-06T01:19:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.553+0000 7f2b85c7f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b801605f0 con 0x7f2b800ce400 2023-07-06T01:19:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7effd700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b800ce400 0x7f2b8015d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b800cdf80 0x7f2b80151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b8015c750 0x7f2b8009aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.559 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b800cdf80 0x7f2b80151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.103:34806/0 (socket says 172.21.15.103:34806) 2023-07-06T01:19:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7f7fe700 1 -- 172.21.15.103:0/1579155554 learned_addr learned my addr 172.21.15.103:0/1579155554 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-07-06T01:19:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7ffff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b8015c750 0x7f2b8009aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47512/0 (socket says 172.21.15.103:47512) 2023-07-06T01:19:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7effd700 1 --2- 172.21.15.103:0/1579155554 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b800ce400 0x7f2b8015d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:44630/0 (socket says 172.21.15.103:44630) 2023-07-06T01:19:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7ffff700 1 -- 172.21.15.103:0/1579155554 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b800ce400 msgr2=0x7f2b8015d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7ffff700 1 --2- 172.21.15.103:0/1579155554 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b800ce400 0x7f2b8015d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7ffff700 1 -- 172.21.15.103:0/1579155554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b800cdf80 msgr2=0x7f2b80151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.560 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7ffff700 1 --2- 172.21.15.103:0/1579155554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b800cdf80 0x7f2b80151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7ffff700 1 -- 172.21.15.103:0/1579155554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b70009b30 con 0x7f2b8015c750 2023-07-06T01:19:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7effd700 1 --2- 172.21.15.103:0/1579155554 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b800ce400 0x7f2b8015d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b7ffff700 1 --2- 172.21.15.103:0/1579155554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b8015c750 0x7f2b8009aa80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c360 tx=0x7f2b7800c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b67fff700 1 -- 172.21.15.103:0/1579155554 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b780052d0 con 0x7f2b8015c750 2023-07-06T01:19:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b67fff700 1 -- 172.21.15.103:0/1579155554 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b78005430 con 0x7f2b8015c750 2023-07-06T01:19:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1579155554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b801608d0 con 0x7f2b8015c750 2023-07-06T01:19:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b67fff700 1 -- 172.21.15.103:0/1579155554 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b780038c0 con 0x7f2b8015c750 2023-07-06T01:19:07.561 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1579155554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8015de70 con 0x7f2b8015c750 2023-07-06T01:19:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b67fff700 1 -- 172.21.15.103:0/1579155554 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2b78003d20 con 0x7f2b8015c750 2023-07-06T01:19:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.554+0000 7f2b67fff700 1 -- 172.21.15.103:0/1579155554 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f2b7801c860 con 0x7f2b8015c750 2023-07-06T01:19:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.555+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1579155554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8009c800 con 0x7f2b8015c750 2023-07-06T01:19:07.562 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.557+0000 7f2b67fff700 1 -- 172.21.15.103:0/1579155554 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f2b8009c800 con 0x7f2b8015c750 2023-07-06T01:19:07.563 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:07.560+0000 7f786f557200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T01:19:07.587 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:07.584+0000 7fde73547200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T01:19:07.643 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.641+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1579155554 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a447ebab-c06d-4d60-a53c-e6847032aeba", "id": 2} v 0) v1 -- 0x7f2b80160df0 con 0x7f2b8015c750 2023-07-06T01:19:07.646 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.644+0000 7f2b67fff700 1 -- 172.21.15.103:0/1579155554 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a447ebab-c06d-4d60-a53c-e6847032aeba", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2b78017070 con 0x7f2b8015c750 2023-07-06T01:19:07.646 INFO:teuthology.orchestra.run.smithi103.stdout:2 2023-07-06T01:19:07.647 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.645+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1579155554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b8015c750 msgr2=0x7f2b8009aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.647 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.645+0000 7f2b85c7f700 1 --2- 172.21.15.103:0/1579155554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b8015c750 0x7f2b8009aa80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c360 tx=0x7f2b7800c670 comp rx=0 tx=0).stop 2023-07-06T01:19:07.648 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.645+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1579155554 shutdown_connections 2023-07-06T01:19:07.648 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.645+0000 7f2b85c7f700 1 --2- 172.21.15.103:0/1579155554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b8015c750 0x7f2b8009aa80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.648 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.645+0000 7f2b85c7f700 1 --2- 172.21.15.103:0/1579155554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b800cdf80 0x7f2b80151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.645+0000 7f2b85c7f700 1 --2- 172.21.15.103:0/1579155554 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b800ce400 0x7f2b8015d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.645+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1579155554 >> 172.21.15.103:0/1579155554 conn(0x7f2b800999f0 msgr2=0x7f2b800ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.645+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1579155554 shutdown_connections 2023-07-06T01:19:07.649 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.645+0000 7f2b85c7f700 1 -- 172.21.15.103:0/1579155554 wait complete. 2023-07-06T01:19:07.659 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new 046c24dd-f90c-4dab-b24b-db82361d3c52 3 2023-07-06T01:19:07.672 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:07.671+0000 7fde73547200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T01:19:07.783 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.781+0000 7ff4945fc700 1 Processor -- start 2023-07-06T01:19:07.783 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.781+0000 7ff4945fc700 1 -- start start 2023-07-06T01:19:07.784 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.782+0000 7ff4945fc700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff48c0af2a0 0x7ff48c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.785 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.782+0000 7ff4945fc700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c0af670 con 0x7ff48c0cc320 2023-07-06T01:19:07.785 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.782+0000 7ff4945fc700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c06d030 con 0x7ff48c06a000 2023-07-06T01:19:07.785 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.782+0000 7ff4945fc700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48c0cd560 con 0x7ff48c0af2a0 2023-07-06T01:19:07.785 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.782+0000 7ff492398700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff48c0af2a0 0x7ff48c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.785 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.782+0000 7ff492398700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff48c0af2a0 0x7ff48c06caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47514/0 (socket says 172.21.15.103:47514) 2023-07-06T01:19:07.785 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.782+0000 7ff492398700 1 -- 172.21.15.103:0/3137080578 learned_addr learned my addr 172.21.15.103:0/3137080578 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-07-06T01:19:07.785 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.782+0000 7ff491b97700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7ff48c06a000 0x7ff48c0cbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:52354/0 (socket says 172.21.15.103:52354) 2023-07-06T01:19:07.786 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.782+0000 7ff492398700 1 -- 172.21.15.103:0/3137080578 >> v1:172.21.15.103:6789/0 conn(0x7ff48c06a000 legacy=0x7ff48c0cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.786 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.782+0000 7ff492398700 1 -- 172.21.15.103:0/3137080578 >> v1:172.21.15.22:6789/0 conn(0x7ff48c0cc320 legacy=0x7ff48c160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.786 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.782+0000 7ff492398700 1 -- 172.21.15.103:0/3137080578 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c0cc760 con 0x7ff48c0af2a0 2023-07-06T01:19:07.786 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.783+0000 7ff492398700 1 --2- 172.21.15.103:0/3137080578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c0af2a0 0x7ff48c06caf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4880098f0 tx=0x7ff488009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a59c79d0f1a6da8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:07.786 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.783+0000 7ff491396700 1 -- 172.21.15.103:0/3137080578 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48800b8a0 con 0x7ff48c0af2a0 2023-07-06T01:19:07.787 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.783+0000 7ff491396700 1 -- 172.21.15.103:0/3137080578 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48800ba00 con 0x7ff48c0af2a0 2023-07-06T01:19:07.787 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.783+0000 7ff491396700 1 -- 172.21.15.103:0/3137080578 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48800bd90 con 0x7ff48c0af2a0 2023-07-06T01:19:07.789 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.783+0000 7ff4945fc700 1 -- 172.21.15.103:0/3137080578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c0af2a0 msgr2=0x7ff48c06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.789 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.783+0000 7ff4945fc700 1 --2- 172.21.15.103:0/3137080578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c0af2a0 0x7ff48c06caf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4880098f0 tx=0x7ff488009c00 comp rx=0 tx=0).stop 2023-07-06T01:19:07.789 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.783+0000 7ff4945fc700 1 -- 172.21.15.103:0/3137080578 shutdown_connections 2023-07-06T01:19:07.790 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.783+0000 7ff4945fc700 1 --2- 172.21.15.103:0/3137080578 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c0af2a0 0x7ff48c06caf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.790 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.783+0000 7ff4945fc700 1 -- 172.21.15.103:0/3137080578 >> 172.21.15.103:0/3137080578 conn(0x7ff48c0b1890 msgr2=0x7ff48c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:07.790 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.783+0000 7ff4945fc700 1 -- 172.21.15.103:0/3137080578 shutdown_connections 2023-07-06T01:19:07.790 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.783+0000 7ff4945fc700 1 -- 172.21.15.103:0/3137080578 wait complete. 2023-07-06T01:19:07.790 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.785+0000 7ff4945fc700 1 Processor -- start 2023-07-06T01:19:07.790 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.785+0000 7ff4945fc700 1 -- start start 2023-07-06T01:19:07.791 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.785+0000 7ff4945fc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff48c06a000 0x7ff48c1f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.791 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.785+0000 7ff4945fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff48c0af2a0 0x7ff48c0c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.791 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.785+0000 7ff4945fc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c0cc320 0x7ff48c0b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:07.791 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.785+0000 7ff4945fc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0b47f0 con 0x7ff48c0af2a0 2023-07-06T01:19:07.791 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.785+0000 7ff4945fc700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0b3be0 con 0x7ff48c0cc320 2023-07-06T01:19:07.791 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.785+0000 7ff4945fc700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0b3ec0 con 0x7ff48c06a000 2023-07-06T01:19:07.791 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.785+0000 7ff492398700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff48c06a000 0x7ff48c1f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.792 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.785+0000 7ff492b99700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c0cc320 0x7ff48c0b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.792 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff492398700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff48c06a000 0x7ff48c1f5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:44640/0 (socket says 172.21.15.103:44640) 2023-07-06T01:19:07.792 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff492398700 1 -- 172.21.15.103:0/1616536936 learned_addr learned my addr 172.21.15.103:0/1616536936 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-07-06T01:19:07.792 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff491b97700 1 --2- 172.21.15.103:0/1616536936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff48c0af2a0 0x7ff48c0c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:07.792 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff492398700 1 -- 172.21.15.103:0/1616536936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c0cc320 msgr2=0x7ff48c0b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.792 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff492398700 1 --2- 172.21.15.103:0/1616536936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c0cc320 0x7ff48c0b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.792 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff492398700 1 -- 172.21.15.103:0/1616536936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff48c0af2a0 msgr2=0x7ff48c0c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.793 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff492398700 1 --2- 172.21.15.103:0/1616536936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff48c0af2a0 0x7ff48c0c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.793 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff492398700 1 -- 172.21.15.103:0/1616536936 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff488009550 con 0x7ff48c06a000 2023-07-06T01:19:07.793 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff492b99700 1 --2- 172.21.15.103:0/1616536936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c0cc320 0x7ff48c0b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:07.793 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff492398700 1 --2- 172.21.15.103:0/1616536936 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff48c06a000 0x7ff48c1f5380 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff488000c00 tx=0x7ff488004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:07.793 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff482ffd700 1 -- 172.21.15.103:0/1616536936 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488004380 con 0x7ff48c06a000 2023-07-06T01:19:07.793 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff482ffd700 1 -- 172.21.15.103:0/1616536936 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48801f070 con 0x7ff48c06a000 2023-07-06T01:19:07.793 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff4945fc700 1 -- 172.21.15.103:0/1616536936 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c0b41a0 con 0x7ff48c06a000 2023-07-06T01:19:07.794 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff482ffd700 1 -- 172.21.15.103:0/1616536936 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4880036a0 con 0x7ff48c06a000 2023-07-06T01:19:07.794 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff4945fc700 1 -- 172.21.15.103:0/1616536936 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c1f58c0 con 0x7ff48c06a000 2023-07-06T01:19:07.794 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff482ffd700 1 -- 172.21.15.103:0/1616536936 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff488021030 con 0x7ff48c06a000 2023-07-06T01:19:07.794 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.786+0000 7ff482ffd700 1 -- 172.21.15.103:0/1616536936 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7ff48801ad40 con 0x7ff48c06a000 2023-07-06T01:19:07.794 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.787+0000 7ff4945fc700 1 -- 172.21.15.103:0/1616536936 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48c15ca90 con 0x7ff48c06a000 2023-07-06T01:19:07.795 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.789+0000 7ff482ffd700 1 -- 172.21.15.103:0/1616536936 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff48c15ca90 con 0x7ff48c06a000 2023-07-06T01:19:07.874 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.872+0000 7ff4945fc700 1 -- 172.21.15.103:0/1616536936 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd new", "uuid": "046c24dd-f90c-4dab-b24b-db82361d3c52", "id": 3} v 0) v1 -- 0x7ff48c0744b0 con 0x7ff48c06a000 2023-07-06T01:19:07.877 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.875+0000 7ff482ffd700 1 -- 172.21.15.103:0/1616536936 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "046c24dd-f90c-4dab-b24b-db82361d3c52", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff488015020 con 0x7ff48c06a000 2023-07-06T01:19:07.877 INFO:teuthology.orchestra.run.smithi103.stdout:3 2023-07-06T01:19:07.879 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.876+0000 7ff4945fc700 1 -- 172.21.15.103:0/1616536936 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff48c06a000 msgr2=0x7ff48c1f5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:07.879 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.876+0000 7ff4945fc700 1 --2- 172.21.15.103:0/1616536936 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff48c06a000 0x7ff48c1f5380 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff488000c00 tx=0x7ff488004750 comp rx=0 tx=0).stop 2023-07-06T01:19:07.879 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.876+0000 7ff4945fc700 1 -- 172.21.15.103:0/1616536936 shutdown_connections 2023-07-06T01:19:07.880 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.876+0000 7ff4945fc700 1 --2- 172.21.15.103:0/1616536936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff48c0cc320 0x7ff48c0b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.880 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.876+0000 7ff4945fc700 1 --2- 172.21.15.103:0/1616536936 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff48c06a000 0x7ff48c1f5380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.880 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.876+0000 7ff4945fc700 1 --2- 172.21.15.103:0/1616536936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff48c0af2a0 0x7ff48c0c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:07.880 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.876+0000 7ff4945fc700 1 -- 172.21.15.103:0/1616536936 >> 172.21.15.103:0/1616536936 conn(0x7ff48c0b1890 msgr2=0x7ff48c15c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:07.881 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.876+0000 7ff4945fc700 1 -- 172.21.15.103:0/1616536936 shutdown_connections 2023-07-06T01:19:07.881 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:07.876+0000 7ff4945fc700 1 -- 172.21.15.103:0/1616536936 wait complete. 2023-07-06T01:19:07.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new dd31524d-cdf4-42da-a0d4-14962f66fbea 4 2023-07-06T01:19:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.008+0000 7f8defa2f700 1 Processor -- start 2023-07-06T01:19:08.010 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.008+0000 7f8defa2f700 1 -- start start 2023-07-06T01:19:08.012 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.009+0000 7f8defa2f700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8de815a610 0x7f8de806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.009+0000 7f8defa2f700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f8de815eae0 0x7f8de806c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.009+0000 7f8defa2f700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8de806c730 0x7f8de806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.009+0000 7f8defa2f700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de815a9e0 con 0x7f8de815a610 2023-07-06T01:19:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.009+0000 7f8defa2f700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de806cbb0 con 0x7f8de815eae0 2023-07-06T01:19:08.013 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.009+0000 7f8defa2f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de8071f30 con 0x7f8de806c730 2023-07-06T01:19:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.009+0000 7f8decfca700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f8de815eae0 0x7f8de806c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.009+0000 7f8dedfcc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8de806c730 0x7f8de806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.009+0000 7f8decfca700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f8de815eae0 0x7f8de806c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47528/0 (socket says 172.21.15.103:47528) 2023-07-06T01:19:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.009+0000 7f8decfca700 1 -- 172.21.15.103:0/1059637996 learned_addr learned my addr 172.21.15.103:0/1059637996 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-07-06T01:19:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.009+0000 7f8ded7cb700 1 --2- 172.21.15.103:0/1059637996 >> v2:172.21.15.22:3300/0 conn(0x7f8de815a610 0x7f8de806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.010+0000 7f8dedfcc700 1 -- 172.21.15.103:0/1059637996 >> v2:172.21.15.103:3300/0 conn(0x7f8de815eae0 msgr2=0x7f8de806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:08.014 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.010+0000 7f8dedfcc700 1 --2- 172.21.15.103:0/1059637996 >> v2:172.21.15.103:3300/0 conn(0x7f8de815eae0 0x7f8de806c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.010+0000 7f8dedfcc700 1 -- 172.21.15.103:0/1059637996 >> v2:172.21.15.22:3300/0 conn(0x7f8de815a610 msgr2=0x7f8de806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.010+0000 7f8dedfcc700 1 --2- 172.21.15.103:0/1059637996 >> v2:172.21.15.22:3300/0 conn(0x7f8de815a610 0x7f8de806b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.010+0000 7f8dedfcc700 1 -- 172.21.15.103:0/1059637996 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de8071320 con 0x7f8de806c730 2023-07-06T01:19:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.010+0000 7f8decfca700 1 --2- 172.21.15.103:0/1059637996 >> v2:172.21.15.103:3300/0 conn(0x7f8de815eae0 0x7f8de806c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.010+0000 7f8ded7cb700 1 --2- 172.21.15.103:0/1059637996 >> v2:172.21.15.22:3300/0 conn(0x7f8de815a610 0x7f8de806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.010+0000 7f8dedfcc700 1 --2- 172.21.15.103:0/1059637996 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8de806c730 0x7f8de806a000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8de80ae810 tx=0x7f8de000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7adf5181cb44dbdc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:08.015 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.010+0000 7f8ddffff700 1 -- 172.21.15.103:0/1059637996 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de000e070 con 0x7f8de806c730 2023-07-06T01:19:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.010+0000 7f8ddffff700 1 -- 172.21.15.103:0/1059637996 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de000bb20 con 0x7f8de806c730 2023-07-06T01:19:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.010+0000 7f8ddffff700 1 -- 172.21.15.103:0/1059637996 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0012430 con 0x7f8de806c730 2023-07-06T01:19:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.011+0000 7f8defa2f700 1 -- 172.21.15.103:0/1059637996 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8de806c730 msgr2=0x7f8de806a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.011+0000 7f8defa2f700 1 --2- 172.21.15.103:0/1059637996 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8de806c730 0x7f8de806a000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8de80ae810 tx=0x7f8de000b330 comp rx=0 tx=0).stop 2023-07-06T01:19:08.016 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.011+0000 7f8defa2f700 1 -- 172.21.15.103:0/1059637996 shutdown_connections 2023-07-06T01:19:08.017 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.011+0000 7f8defa2f700 1 --2- 172.21.15.103:0/1059637996 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8de806c730 0x7f8de806a000 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.017 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.011+0000 7f8defa2f700 1 --2- 172.21.15.103:0/1059637996 >> v2:172.21.15.22:3300/0 conn(0x7f8de815a610 0x7f8de806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.017 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.011+0000 7f8defa2f700 1 --2- 172.21.15.103:0/1059637996 >> v2:172.21.15.103:3300/0 conn(0x7f8de815eae0 0x7f8de806c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.017 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.011+0000 7f8defa2f700 1 -- 172.21.15.103:0/1059637996 >> 172.21.15.103:0/1059637996 conn(0x7f8de80ccb00 msgr2=0x7f8de80cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:08.017 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.011+0000 7f8defa2f700 1 -- 172.21.15.103:0/1059637996 shutdown_connections 2023-07-06T01:19:08.017 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.011+0000 7f8defa2f700 1 -- 172.21.15.103:0/1059637996 wait complete. 2023-07-06T01:19:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.012+0000 7f8defa2f700 1 Processor -- start 2023-07-06T01:19:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.012+0000 7f8defa2f700 1 -- start start 2023-07-06T01:19:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8defa2f700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8de806c730 0x7f8de80c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8defa2f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de815a610 0x7f8de81e5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.020 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8defa2f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8de815eae0 0x7f8de814e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8defa2f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80c4360 con 0x7f8de815eae0 2023-07-06T01:19:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8defa2f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80c3750 con 0x7f8de815a610 2023-07-06T01:19:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8defa2f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80c3a30 con 0x7f8de806c730 2023-07-06T01:19:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8decfca700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de815a610 0x7f8de81e5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8decfca700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de815a610 0x7f8de81e5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47534/0 (socket says 172.21.15.103:47534) 2023-07-06T01:19:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8decfca700 1 -- 172.21.15.103:0/4008210995 learned_addr learned my addr 172.21.15.103:0/4008210995 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-07-06T01:19:08.021 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8ded7cb700 1 --2- 172.21.15.103:0/4008210995 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8de806c730 0x7f8de80c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.022 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8dedfcc700 1 --2- 172.21.15.103:0/4008210995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8de815eae0 0x7f8de814e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.022 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8decfca700 1 -- 172.21.15.103:0/4008210995 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8de806c730 msgr2=0x7f8de80c2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.022 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8decfca700 1 --2- 172.21.15.103:0/4008210995 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8de806c730 0x7f8de80c2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.022 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8decfca700 1 -- 172.21.15.103:0/4008210995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8de815eae0 msgr2=0x7f8de814e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.022 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8decfca700 1 --2- 172.21.15.103:0/4008210995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8de815eae0 0x7f8de814e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.022 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8decfca700 1 -- 172.21.15.103:0/4008210995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd8009550 con 0x7f8de815a610 2023-07-06T01:19:08.023 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8ded7cb700 1 --2- 172.21.15.103:0/4008210995 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8de806c730 0x7f8de80c2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:08.023 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8decfca700 1 --2- 172.21.15.103:0/4008210995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de815a610 0x7f8de81e5a70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800e9b0 tx=0x7f8dd800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:08.023 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8dddffb700 1 -- 172.21.15.103:0/4008210995 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800ee50 con 0x7f8de815a610 2023-07-06T01:19:08.023 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8dddffb700 1 -- 172.21.15.103:0/4008210995 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd800c850 con 0x7f8de815a610 2023-07-06T01:19:08.023 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8defa2f700 1 -- 172.21.15.103:0/4008210995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de0009a70 con 0x7f8de815a610 2023-07-06T01:19:08.023 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.013+0000 7f8dddffb700 1 -- 172.21.15.103:0/4008210995 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800cbc0 con 0x7f8de815a610 2023-07-06T01:19:08.023 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.014+0000 7f8defa2f700 1 -- 172.21.15.103:0/4008210995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de80c3ff0 con 0x7f8de815a610 2023-07-06T01:19:08.024 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.014+0000 7f8dddffb700 1 -- 172.21.15.103:0/4008210995 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8dd8005d20 con 0x7f8de815a610 2023-07-06T01:19:08.024 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.014+0000 7f8dddffb700 1 -- 172.21.15.103:0/4008210995 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f8dd8006750 con 0x7f8de815a610 2023-07-06T01:19:08.024 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.015+0000 7f8defa2f700 1 -- 172.21.15.103:0/4008210995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de80700e0 con 0x7f8de815a610 2023-07-06T01:19:08.024 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.017+0000 7f8dddffb700 1 -- 172.21.15.103:0/4008210995 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f8dd8021060 con 0x7f8de815a610 2023-07-06T01:19:08.024 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:08.021+0000 7f786f557200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T01:19:08.101 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.099+0000 7f8defa2f700 1 -- 172.21.15.103:0/4008210995 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dd31524d-cdf4-42da-a0d4-14962f66fbea", "id": 4} v 0) v1 -- 0x7f8de80c44d0 con 0x7f8de815a610 2023-07-06T01:19:08.106 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.105+0000 7f8dddffb700 1 -- 172.21.15.103:0/4008210995 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd31524d-cdf4-42da-a0d4-14962f66fbea", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8dd8017070 con 0x7f8de815a610 2023-07-06T01:19:08.107 INFO:teuthology.orchestra.run.smithi103.stdout:4 2023-07-06T01:19:08.108 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.106+0000 7f8defa2f700 1 -- 172.21.15.103:0/4008210995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de815a610 msgr2=0x7f8de81e5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.108 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.106+0000 7f8defa2f700 1 --2- 172.21.15.103:0/4008210995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de815a610 0x7f8de81e5a70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800e9b0 tx=0x7f8dd800ecc0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.108 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.106+0000 7f8defa2f700 1 -- 172.21.15.103:0/4008210995 shutdown_connections 2023-07-06T01:19:08.108 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.106+0000 7f8defa2f700 1 --2- 172.21.15.103:0/4008210995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8de815eae0 0x7f8de814e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.108 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.106+0000 7f8defa2f700 1 --2- 172.21.15.103:0/4008210995 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8de806c730 0x7f8de80c2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.109 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.106+0000 7f8defa2f700 1 --2- 172.21.15.103:0/4008210995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8de815a610 0x7f8de81e5a70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.109 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.106+0000 7f8defa2f700 1 -- 172.21.15.103:0/4008210995 >> 172.21.15.103:0/4008210995 conn(0x7f8de80ccb00 msgr2=0x7f8de806b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:08.109 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.106+0000 7f8defa2f700 1 -- 172.21.15.103:0/4008210995 shutdown_connections 2023-07-06T01:19:08.109 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.106+0000 7f8defa2f700 1 -- 172.21.15.103:0/4008210995 wait complete. 2023-07-06T01:19:08.121 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new 7902da34-cb61-4494-a428-e23f242846bf 5 2023-07-06T01:19:08.127 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:08.126+0000 7fde73547200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T01:19:08.242 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.240+0000 7fa510a92700 1 Processor -- start 2023-07-06T01:19:08.242 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.240+0000 7fa510a92700 1 -- start start 2023-07-06T01:19:08.244 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa510a92700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa50c15a780 0x7fa50c06a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.244 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa510a92700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa50c15f100 0x7fa50c06a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.244 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa510a92700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa50c0cfbf0 0x7fa50c0d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.244 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa510a92700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50c15e3d0 con 0x7fa50c0cfbf0 2023-07-06T01:19:08.245 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa510a92700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50c15ec70 con 0x7fa50c15f100 2023-07-06T01:19:08.245 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa510a92700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0d0920 con 0x7fa50c15a780 2023-07-06T01:19:08.245 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa509d9b700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa50c15f100 0x7fa50c06a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.245 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa50a59c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa50c15a780 0x7fa50c06a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.245 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa509d9b700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa50c15f100 0x7fa50c06a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47538/0 (socket says 172.21.15.103:47538) 2023-07-06T01:19:08.245 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa509d9b700 1 -- 172.21.15.103:0/1928610646 learned_addr learned my addr 172.21.15.103:0/1928610646 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-07-06T01:19:08.245 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa50ad9d700 1 --2- 172.21.15.103:0/1928610646 >> v2:172.21.15.22:3300/0 conn(0x7fa50c0cfbf0 0x7fa50c0d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.246 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa509d9b700 1 -- 172.21.15.103:0/1928610646 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa50c15a780 msgr2=0x7fa50c06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:08.246 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa509d9b700 1 --2- 172.21.15.103:0/1928610646 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa50c15a780 0x7fa50c06a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.246 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa509d9b700 1 -- 172.21.15.103:0/1928610646 >> v2:172.21.15.22:3300/0 conn(0x7fa50c0cfbf0 msgr2=0x7fa50c0d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:08.246 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa509d9b700 1 --2- 172.21.15.103:0/1928610646 >> v2:172.21.15.22:3300/0 conn(0x7fa50c0cfbf0 0x7fa50c0d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.246 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa509d9b700 1 -- 172.21.15.103:0/1928610646 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c15e620 con 0x7fa50c15f100 2023-07-06T01:19:08.246 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa50ad9d700 1 --2- 172.21.15.103:0/1928610646 >> v2:172.21.15.22:3300/0 conn(0x7fa50c0cfbf0 0x7fa50c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:08.246 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.241+0000 7fa509d9b700 1 --2- 172.21.15.103:0/1928610646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa50c15f100 0x7fa50c06a540 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa50c06c1a0 tx=0x7fa500009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3500486f7726b865 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:08.246 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.242+0000 7fa50959a700 1 -- 172.21.15.103:0/1928610646 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500011830 con 0x7fa50c15f100 2023-07-06T01:19:08.247 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.242+0000 7fa50959a700 1 -- 172.21.15.103:0/1928610646 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa500011990 con 0x7fa50c15f100 2023-07-06T01:19:08.247 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.242+0000 7fa50959a700 1 -- 172.21.15.103:0/1928610646 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500011d20 con 0x7fa50c15f100 2023-07-06T01:19:08.247 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.242+0000 7fa510a92700 1 -- 172.21.15.103:0/1928610646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa50c15f100 msgr2=0x7fa50c06a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.247 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.242+0000 7fa510a92700 1 --2- 172.21.15.103:0/1928610646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa50c15f100 0x7fa50c06a540 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa50c06c1a0 tx=0x7fa500009bd0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.247 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.242+0000 7fa510a92700 1 -- 172.21.15.103:0/1928610646 shutdown_connections 2023-07-06T01:19:08.247 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.242+0000 7fa510a92700 1 --2- 172.21.15.103:0/1928610646 >> v2:172.21.15.22:3300/0 conn(0x7fa50c0cfbf0 0x7fa50c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.247 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.242+0000 7fa510a92700 1 --2- 172.21.15.103:0/1928610646 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa50c15a780 0x7fa50c06a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.248 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.242+0000 7fa510a92700 1 --2- 172.21.15.103:0/1928610646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa50c15f100 0x7fa50c06a540 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.248 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.242+0000 7fa510a92700 1 -- 172.21.15.103:0/1928610646 >> 172.21.15.103:0/1928610646 conn(0x7fa50c06b110 msgr2=0x7fa50c06b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:08.248 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.242+0000 7fa510a92700 1 -- 172.21.15.103:0/1928610646 shutdown_connections 2023-07-06T01:19:08.248 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.242+0000 7fa510a92700 1 -- 172.21.15.103:0/1928610646 wait complete. 2023-07-06T01:19:08.250 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.244+0000 7fa510a92700 1 Processor -- start 2023-07-06T01:19:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.244+0000 7fa510a92700 1 -- start start 2023-07-06T01:19:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.244+0000 7fa510a92700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa50c0cfbf0 0x7fa50c0b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.244+0000 7fa510a92700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa50c15a780 0x7fa50c15c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.244+0000 7fa510a92700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa50c15f100 0x7fa50c151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.244+0000 7fa510a92700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c0ac870 con 0x7fa50c0cfbf0 2023-07-06T01:19:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.244+0000 7fa510a92700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c15cdd0 con 0x7fa50c15f100 2023-07-06T01:19:08.251 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.244+0000 7fa510a92700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c15d070 con 0x7fa50c15a780 2023-07-06T01:19:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.244+0000 7fa50ad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa50c15f100 0x7fa50c151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.244+0000 7fa509d9b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa50c15a780 0x7fa50c15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.244+0000 7fa50ad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa50c15f100 0x7fa50c151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47548/0 (socket says 172.21.15.103:47548) 2023-07-06T01:19:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.244+0000 7fa50ad9d700 1 -- 172.21.15.103:0/3404720811 learned_addr learned my addr 172.21.15.103:0/3404720811 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-07-06T01:19:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.244+0000 7fa50a59c700 1 --2- 172.21.15.103:0/3404720811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa50c0cfbf0 0x7fa50c0b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.252 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa509d9b700 1 -- 172.21.15.103:0/3404720811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa50c15f100 msgr2=0x7fa50c151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.253 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa509d9b700 1 --2- 172.21.15.103:0/3404720811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa50c15f100 0x7fa50c151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.253 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa509d9b700 1 -- 172.21.15.103:0/3404720811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa50c0cfbf0 msgr2=0x7fa50c0b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.253 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa509d9b700 1 --2- 172.21.15.103:0/3404720811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa50c0cfbf0 0x7fa50c0b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.253 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa509d9b700 1 -- 172.21.15.103:0/3404720811 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa500009550 con 0x7fa50c15a780 2023-07-06T01:19:08.253 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa50ad9d700 1 --2- 172.21.15.103:0/3404720811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa50c15f100 0x7fa50c151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:08.253 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa50a59c700 1 --2- 172.21.15.103:0/3404720811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa50c0cfbf0 0x7fa50c0b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:08.254 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa509d9b700 1 --2- 172.21.15.103:0/3404720811 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa50c15a780 0x7fa50c15c890 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa500016000 tx=0x7fa500004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:08.254 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa4faffd700 1 -- 172.21.15.103:0/3404720811 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500004450 con 0x7fa50c15a780 2023-07-06T01:19:08.254 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa4faffd700 1 -- 172.21.15.103:0/3404720811 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50001e070 con 0x7fa50c15a780 2023-07-06T01:19:08.254 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa4faffd700 1 -- 172.21.15.103:0/3404720811 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5000038e0 con 0x7fa50c15a780 2023-07-06T01:19:08.255 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa510a92700 1 -- 172.21.15.103:0/3404720811 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50c15d350 con 0x7fa50c15a780 2023-07-06T01:19:08.255 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa510a92700 1 -- 172.21.15.103:0/3404720811 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50c0acda0 con 0x7fa50c15a780 2023-07-06T01:19:08.255 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa4faffd700 1 -- 172.21.15.103:0/3404720811 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa500003d40 con 0x7fa50c15a780 2023-07-06T01:19:08.256 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.245+0000 7fa4faffd700 1 -- 172.21.15.103:0/3404720811 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fa50002db30 con 0x7fa50c15a780 2023-07-06T01:19:08.256 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.246+0000 7fa510a92700 1 -- 172.21.15.103:0/3404720811 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50c0744b0 con 0x7fa50c15a780 2023-07-06T01:19:08.256 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.250+0000 7fa4faffd700 1 -- 172.21.15.103:0/3404720811 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fa500020070 con 0x7fa50c15a780 2023-07-06T01:19:08.335 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.333+0000 7fa510a92700 1 -- 172.21.15.103:0/3404720811 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7902da34-cb61-4494-a428-e23f242846bf", "id": 5} v 0) v1 -- 0x7fa50c0cd5f0 con 0x7fa50c15a780 2023-07-06T01:19:08.343 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.341+0000 7fa4faffd700 1 -- 172.21.15.103:0/3404720811 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7902da34-cb61-4494-a428-e23f242846bf", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa500015020 con 0x7fa50c15a780 2023-07-06T01:19:08.343 INFO:teuthology.orchestra.run.smithi103.stdout:5 2023-07-06T01:19:08.344 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.342+0000 7fa510a92700 1 -- 172.21.15.103:0/3404720811 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa50c15a780 msgr2=0x7fa50c15c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.344 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.342+0000 7fa510a92700 1 --2- 172.21.15.103:0/3404720811 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa50c15a780 0x7fa50c15c890 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa500016000 tx=0x7fa500004750 comp rx=0 tx=0).stop 2023-07-06T01:19:08.344 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.342+0000 7fa510a92700 1 -- 172.21.15.103:0/3404720811 shutdown_connections 2023-07-06T01:19:08.344 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.342+0000 7fa510a92700 1 --2- 172.21.15.103:0/3404720811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa50c15f100 0x7fa50c151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.345 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.342+0000 7fa510a92700 1 --2- 172.21.15.103:0/3404720811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa50c0cfbf0 0x7fa50c0b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.345 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.342+0000 7fa510a92700 1 --2- 172.21.15.103:0/3404720811 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa50c15a780 0x7fa50c15c890 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.345 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.342+0000 7fa510a92700 1 -- 172.21.15.103:0/3404720811 >> 172.21.15.103:0/3404720811 conn(0x7fa50c06b110 msgr2=0x7fa50c159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:08.345 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.342+0000 7fa510a92700 1 -- 172.21.15.103:0/3404720811 shutdown_connections 2023-07-06T01:19:08.345 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.342+0000 7fa510a92700 1 -- 172.21.15.103:0/3404720811 wait complete. 2023-07-06T01:19:08.355 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new f9d5652b-3c91-45a7-b98a-a56a76ed9247 6 2023-07-06T01:19:08.500 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.499+0000 7fa7710d5700 1 Processor -- start 2023-07-06T01:19:08.501 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.499+0000 7fa7710d5700 1 -- start start 2023-07-06T01:19:08.503 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.499+0000 7fa7710d5700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa76c06a710 0x7fa76c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.503 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.499+0000 7fa7710d5700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa76c0cbc10 0x7fa76c0cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.503 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa7710d5700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76c0af670 con 0x7fa76c0af2a0 2023-07-06T01:19:08.503 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa7710d5700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76c0cd560 con 0x7fa76c06a710 2023-07-06T01:19:08.504 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa7710d5700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c15cab0 con 0x7fa76c0cbc10 2023-07-06T01:19:08.504 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa76a59c700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa76c06a710 0x7fa76c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.504 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa76ad9d700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fa76c0af2a0 0x7fa76c06a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:52388/0 (socket says 172.21.15.103:52388) 2023-07-06T01:19:08.504 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa76ad9d700 1 -- 172.21.15.103:0/1623845085 learned_addr learned my addr 172.21.15.103:0/1623845085 (peer_addr_for_me v1:172.21.15.103:0/0) 2023-07-06T01:19:08.504 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa76b59e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa76c0cbc10 0x7fa76c0cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.504 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa76a59c700 1 -- 172.21.15.103:0/1623845085 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa76c0cbc10 msgr2=0x7fa76c0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:08.504 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa76a59c700 1 --2- 172.21.15.103:0/1623845085 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa76c0cbc10 0x7fa76c0cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.505 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa76a59c700 1 -- 172.21.15.103:0/1623845085 >> v1:172.21.15.103:6789/0 conn(0x7fa76c0af2a0 legacy=0x7fa76c06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.505 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa76a59c700 1 -- 172.21.15.103:0/1623845085 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c06d030 con 0x7fa76c06a710 2023-07-06T01:19:08.505 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa76b59e700 1 --2- 172.21.15.103:0/1623845085 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa76c0cbc10 0x7fa76c0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:08.505 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa76a59c700 1 --2- 172.21.15.103:0/1623845085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76c06a710 0x7fa76c06caf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa760009840 tx=0x7fa760009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd8d671c26a6335d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:08.505 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa769d9b700 1 -- 172.21.15.103:0/1623845085 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760011830 con 0x7fa76c06a710 2023-07-06T01:19:08.505 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa769d9b700 1 -- 172.21.15.103:0/1623845085 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa760011990 con 0x7fa76c06a710 2023-07-06T01:19:08.505 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.500+0000 7fa769d9b700 1 -- 172.21.15.103:0/1623845085 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760011d20 con 0x7fa76c06a710 2023-07-06T01:19:08.506 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.501+0000 7fa7710d5700 1 -- 172.21.15.103:0/1623845085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76c06a710 msgr2=0x7fa76c06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.506 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.501+0000 7fa7710d5700 1 --2- 172.21.15.103:0/1623845085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76c06a710 0x7fa76c06caf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa760009840 tx=0x7fa760009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:08.506 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.501+0000 7fa7710d5700 1 -- 172.21.15.103:0/1623845085 shutdown_connections 2023-07-06T01:19:08.506 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.501+0000 7fa7710d5700 1 --2- 172.21.15.103:0/1623845085 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa76c0cbc10 0x7fa76c0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.506 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.501+0000 7fa7710d5700 1 --2- 172.21.15.103:0/1623845085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76c06a710 0x7fa76c06caf0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.506 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.501+0000 7fa7710d5700 1 -- 172.21.15.103:0/1623845085 >> 172.21.15.103:0/1623845085 conn(0x7fa76c0b1890 msgr2=0x7fa76c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:08.506 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.501+0000 7fa7710d5700 1 -- 172.21.15.103:0/1623845085 shutdown_connections 2023-07-06T01:19:08.506 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.501+0000 7fa7710d5700 1 -- 172.21.15.103:0/1623845085 wait complete. 2023-07-06T01:19:08.508 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.502+0000 7fa7710d5700 1 Processor -- start 2023-07-06T01:19:08.508 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.502+0000 7fa7710d5700 1 -- start start 2023-07-06T01:19:08.509 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.503+0000 7fa7710d5700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76c06a710 0x7fa76c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.509 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.503+0000 7fa7710d5700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa76c0af2a0 0x7fa76c1f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.509 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.503+0000 7fa7710d5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa76c0cbc10 0x7fa76c15e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.509 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.503+0000 7fa7710d5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c1f2890 con 0x7fa76c0cbc10 2023-07-06T01:19:08.510 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.503+0000 7fa7710d5700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c1f2b70 con 0x7fa76c06a710 2023-07-06T01:19:08.510 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.503+0000 7fa7710d5700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c1f2e50 con 0x7fa76c0af2a0 2023-07-06T01:19:08.510 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.503+0000 7fa76ad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76c06a710 0x7fa76c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.510 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.503+0000 7fa76a59c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa76c0af2a0 0x7fa76c1f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.510 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.503+0000 7fa76ad9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76c06a710 0x7fa76c0b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47556/0 (socket says 172.21.15.103:47556) 2023-07-06T01:19:08.510 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.503+0000 7fa76a59c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa76c0af2a0 0x7fa76c1f1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:44678/0 (socket says 172.21.15.103:44678) 2023-07-06T01:19:08.510 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.503+0000 7fa76ad9d700 1 -- 172.21.15.103:0/3915672147 learned_addr learned my addr 172.21.15.103:0/3915672147 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-07-06T01:19:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.503+0000 7fa76b59e700 1 --2- 172.21.15.103:0/3915672147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa76c0cbc10 0x7fa76c15e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.504+0000 7fa76ad9d700 1 -- 172.21.15.103:0/3915672147 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa76c0af2a0 msgr2=0x7fa76c1f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.504+0000 7fa76ad9d700 1 --2- 172.21.15.103:0/3915672147 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa76c0af2a0 0x7fa76c1f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.504+0000 7fa76ad9d700 1 -- 172.21.15.103:0/3915672147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa76c0cbc10 msgr2=0x7fa76c15e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.504+0000 7fa76ad9d700 1 --2- 172.21.15.103:0/3915672147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa76c0cbc10 0x7fa76c15e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.504+0000 7fa76ad9d700 1 -- 172.21.15.103:0/3915672147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764009a70 con 0x7fa76c06a710 2023-07-06T01:19:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.504+0000 7fa76ad9d700 1 --2- 172.21.15.103:0/3915672147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76c06a710 0x7fa76c0b5000 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00c520 tx=0x7fa75c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:08.512 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.504+0000 7fa7537fe700 1 -- 172.21.15.103:0/3915672147 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c005d70 con 0x7fa76c06a710 2023-07-06T01:19:08.512 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.504+0000 7fa7537fe700 1 -- 172.21.15.103:0/3915672147 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c0070d0 con 0x7fa76c06a710 2023-07-06T01:19:08.512 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.504+0000 7fa7710d5700 1 -- 172.21.15.103:0/3915672147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7600094a0 con 0x7fa76c06a710 2023-07-06T01:19:08.512 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.504+0000 7fa7537fe700 1 -- 172.21.15.103:0/3915672147 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c007440 con 0x7fa76c06a710 2023-07-06T01:19:08.512 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.504+0000 7fa7710d5700 1 -- 172.21.15.103:0/3915672147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76c15dff0 con 0x7fa76c06a710 2023-07-06T01:19:08.512 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.504+0000 7fa7537fe700 1 -- 172.21.15.103:0/3915672147 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa75c01c440 con 0x7fa76c06a710 2023-07-06T01:19:08.512 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.504+0000 7fa7537fe700 1 -- 172.21.15.103:0/3915672147 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fa75c01cce0 con 0x7fa76c06a710 2023-07-06T01:19:08.512 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.505+0000 7fa7710d5700 1 -- 172.21.15.103:0/3915672147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76c0744b0 con 0x7fa76c06a710 2023-07-06T01:19:08.513 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.508+0000 7fa7537fe700 1 -- 172.21.15.103:0/3915672147 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fa75c021060 con 0x7fa76c06a710 2023-07-06T01:19:08.595 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.591+0000 7fa7710d5700 1 -- 172.21.15.103:0/3915672147 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f9d5652b-3c91-45a7-b98a-a56a76ed9247", "id": 6} v 0) v1 -- 0x7fa76c0712e0 con 0x7fa76c06a710 2023-07-06T01:19:08.598 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:08.596+0000 7f786f557200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T01:19:08.599 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.597+0000 7fa7537fe700 1 -- 172.21.15.103:0/3915672147 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f9d5652b-3c91-45a7-b98a-a56a76ed9247", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa75c017070 con 0x7fa76c06a710 2023-07-06T01:19:08.599 INFO:teuthology.orchestra.run.smithi103.stdout:6 2023-07-06T01:19:08.600 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.598+0000 7fa7710d5700 1 -- 172.21.15.103:0/3915672147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76c06a710 msgr2=0x7fa76c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.600 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.598+0000 7fa7710d5700 1 --2- 172.21.15.103:0/3915672147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76c06a710 0x7fa76c0b5000 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00c520 tx=0x7fa75c00c830 comp rx=0 tx=0).stop 2023-07-06T01:19:08.600 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.598+0000 7fa7710d5700 1 -- 172.21.15.103:0/3915672147 shutdown_connections 2023-07-06T01:19:08.600 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.598+0000 7fa7710d5700 1 --2- 172.21.15.103:0/3915672147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa76c0cbc10 0x7fa76c15e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.600 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.598+0000 7fa7710d5700 1 --2- 172.21.15.103:0/3915672147 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa76c06a710 0x7fa76c0b5000 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.601 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.598+0000 7fa7710d5700 1 --2- 172.21.15.103:0/3915672147 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa76c0af2a0 0x7fa76c1f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.601 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.598+0000 7fa7710d5700 1 -- 172.21.15.103:0/3915672147 >> 172.21.15.103:0/3915672147 conn(0x7fa76c0b1890 msgr2=0x7fa76c15c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:08.601 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.598+0000 7fa7710d5700 1 -- 172.21.15.103:0/3915672147 shutdown_connections 2023-07-06T01:19:08.601 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.598+0000 7fa7710d5700 1 -- 172.21.15.103:0/3915672147 wait complete. 2023-07-06T01:19:08.612 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new bcacc64c-8ca7-4a3c-954b-99e10ce3f659 7 2023-07-06T01:19:08.722 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:08.721+0000 7fde73547200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T01:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.734+0000 7f24d54c6700 1 Processor -- start 2023-07-06T01:19:08.736 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.734+0000 7f24d54c6700 1 -- start start 2023-07-06T01:19:08.739 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24d54c6700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f24d00af2a0 0x7f24d006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.739 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24d54c6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 0x7f24d006a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.739 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24d54c6700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d00af670 con 0x7f24d006a000 2023-07-06T01:19:08.740 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24d54c6700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d006a9e0 con 0x7f24d00af2a0 2023-07-06T01:19:08.740 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24d54c6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00cd560 con 0x7f24d00cc320 2023-07-06T01:19:08.740 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24cf7fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 0x7f24d006a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.740 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24ceffd700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f24d00af2a0 0x7f24d006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.740 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24cf7fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 0x7f24d006a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:44686/0 (socket says 172.21.15.103:44686) 2023-07-06T01:19:08.740 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24ceffd700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f24d00af2a0 0x7f24d006caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47562/0 (socket says 172.21.15.103:47562) 2023-07-06T01:19:08.741 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24cf7fe700 1 -- 172.21.15.103:0/603395623 learned_addr learned my addr 172.21.15.103:0/603395623 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-07-06T01:19:08.741 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24cf7fe700 1 -- 172.21.15.103:0/603395623 >> v2:172.21.15.103:3300/0 conn(0x7f24d00af2a0 msgr2=0x7f24d006caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:08.741 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24cf7fe700 1 --2- 172.21.15.103:0/603395623 >> v2:172.21.15.103:3300/0 conn(0x7f24d00af2a0 0x7f24d006caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.741 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24cf7fe700 1 -- 172.21.15.103:0/603395623 >> v1:172.21.15.103:6789/0 conn(0x7f24d006a000 legacy=0x7f24d00cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.741 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24cf7fe700 1 -- 172.21.15.103:0/603395623 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d006d030 con 0x7f24d00cc320 2023-07-06T01:19:08.741 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24cdffb700 1 -- 172.21.15.103:0/603395623 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808589016 0 0) 0x7f24d00af670 con 0x7f24d006a000 2023-07-06T01:19:08.742 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.735+0000 7f24ceffd700 1 --2- 172.21.15.103:0/603395623 >> v2:172.21.15.103:3300/0 conn(0x7f24d00af2a0 0x7f24d006caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:08.742 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.736+0000 7f24cf7fe700 1 --2- 172.21.15.103:0/603395623 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 0x7f24d006a4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009e50 tx=0x7f24c800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4da5bc1dc12ccf45 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:08.742 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.736+0000 7f24cdffb700 1 -- 172.21.15.103:0/603395623 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c800e070 con 0x7f24d00cc320 2023-07-06T01:19:08.742 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.736+0000 7f24cdffb700 1 -- 172.21.15.103:0/603395623 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c800b910 con 0x7f24d00cc320 2023-07-06T01:19:08.742 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.736+0000 7f24cdffb700 1 -- 172.21.15.103:0/603395623 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c800bcc0 con 0x7f24d00cc320 2023-07-06T01:19:08.742 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.736+0000 7f24d54c6700 1 -- 172.21.15.103:0/603395623 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 msgr2=0x7f24d006a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.743 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.736+0000 7f24d54c6700 1 --2- 172.21.15.103:0/603395623 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 0x7f24d006a4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009e50 tx=0x7f24c800b330 comp rx=0 tx=0).stop 2023-07-06T01:19:08.743 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.736+0000 7f24d54c6700 1 -- 172.21.15.103:0/603395623 shutdown_connections 2023-07-06T01:19:08.743 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.736+0000 7f24d54c6700 1 --2- 172.21.15.103:0/603395623 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 0x7f24d006a4a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.743 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.736+0000 7f24d54c6700 1 --2- 172.21.15.103:0/603395623 >> v2:172.21.15.103:3300/0 conn(0x7f24d00af2a0 0x7f24d006caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.743 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.736+0000 7f24d54c6700 1 -- 172.21.15.103:0/603395623 >> 172.21.15.103:0/603395623 conn(0x7f24d00b1890 msgr2=0x7f24d00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:08.743 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.736+0000 7f24d54c6700 1 -- 172.21.15.103:0/603395623 shutdown_connections 2023-07-06T01:19:08.743 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.736+0000 7f24d54c6700 1 -- 172.21.15.103:0/603395623 wait complete. 2023-07-06T01:19:08.745 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.738+0000 7f24d54c6700 1 Processor -- start 2023-07-06T01:19:08.746 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.738+0000 7f24d54c6700 1 -- start start 2023-07-06T01:19:08.746 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24d54c6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24d006a000 0x7f24d015f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.746 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24d54c6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d00af2a0 0x7f24d0158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.746 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24d54c6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 0x7f24d00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:08.746 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24d54c6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01595d0 con 0x7f24d00af2a0 2023-07-06T01:19:08.746 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24d54c6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01598b0 con 0x7f24d006a000 2023-07-06T01:19:08.746 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24d54c6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d0159b90 con 0x7f24d00cc320 2023-07-06T01:19:08.746 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24cf7fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 0x7f24d00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.747 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24ceffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24d006a000 0x7f24d015f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.747 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24cf7fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 0x7f24d00b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:44692/0 (socket says 172.21.15.103:44692) 2023-07-06T01:19:08.747 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24cf7fe700 1 -- 172.21.15.103:0/3996836086 learned_addr learned my addr 172.21.15.103:0/3996836086 (peer_addr_for_me v2:172.21.15.103:0/0) 2023-07-06T01:19:08.747 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24ce7fc700 1 --2- 172.21.15.103:0/3996836086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d00af2a0 0x7f24d0158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:08.747 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24ceffd700 1 -- 172.21.15.103:0/3996836086 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 msgr2=0x7f24d00b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.748 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24ceffd700 1 --2- 172.21.15.103:0/3996836086 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 0x7f24d00b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.748 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24ceffd700 1 -- 172.21.15.103:0/3996836086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d00af2a0 msgr2=0x7f24d0158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.748 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24ceffd700 1 --2- 172.21.15.103:0/3996836086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d00af2a0 0x7f24d0158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.748 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24ceffd700 1 -- 172.21.15.103:0/3996836086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c00094a0 con 0x7f24d006a000 2023-07-06T01:19:08.749 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24cf7fe700 1 --2- 172.21.15.103:0/3996836086 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 0x7f24d00b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:08.749 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24ceffd700 1 --2- 172.21.15.103:0/3996836086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24d006a000 0x7f24d015f4d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f24c000e9b0 tx=0x7f24c000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:08.749 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24b77fe700 1 -- 172.21.15.103:0/3996836086 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c000ee50 con 0x7f24d006a000 2023-07-06T01:19:08.749 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24b77fe700 1 -- 172.21.15.103:0/3996836086 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c000c850 con 0x7f24d006a000 2023-07-06T01:19:08.750 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24d54c6700 1 -- 172.21.15.103:0/3996836086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c8009b30 con 0x7f24d006a000 2023-07-06T01:19:08.750 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24b77fe700 1 -- 172.21.15.103:0/3996836086 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c000cc00 con 0x7f24d006a000 2023-07-06T01:19:08.750 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.739+0000 7f24d54c6700 1 -- 172.21.15.103:0/3996836086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d015a150 con 0x7f24d006a000 2023-07-06T01:19:08.750 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.740+0000 7f24b77fe700 1 -- 172.21.15.103:0/3996836086 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f24c0005d20 con 0x7f24d006a000 2023-07-06T01:19:08.750 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.740+0000 7f24b77fe700 1 -- 172.21.15.103:0/3996836086 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f24c0006830 con 0x7f24d006a000 2023-07-06T01:19:08.750 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.740+0000 7f24d54c6700 1 -- 172.21.15.103:0/3996836086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d015a4c0 con 0x7f24d006a000 2023-07-06T01:19:08.750 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.743+0000 7f24b77fe700 1 -- 172.21.15.103:0/3996836086 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f24c0021060 con 0x7f24d006a000 2023-07-06T01:19:08.762 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:08.760+0000 7f786f557200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T01:19:08.827 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.825+0000 7f24d54c6700 1 -- 172.21.15.103:0/3996836086 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bcacc64c-8ca7-4a3c-954b-99e10ce3f659", "id": 7} v 0) v1 -- 0x7f24d015ca90 con 0x7f24d006a000 2023-07-06T01:19:08.835 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.833+0000 7f24b77fe700 1 -- 172.21.15.103:0/3996836086 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bcacc64c-8ca7-4a3c-954b-99e10ce3f659", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f24d015ca90 con 0x7f24d006a000 2023-07-06T01:19:08.836 INFO:teuthology.orchestra.run.smithi103.stdout:7 2023-07-06T01:19:08.837 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.834+0000 7f24d54c6700 1 -- 172.21.15.103:0/3996836086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24d006a000 msgr2=0x7f24d015f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:08.837 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.834+0000 7f24d54c6700 1 --2- 172.21.15.103:0/3996836086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24d006a000 0x7f24d015f4d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f24c000e9b0 tx=0x7f24c000ecc0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.838 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.834+0000 7f24d54c6700 1 -- 172.21.15.103:0/3996836086 shutdown_connections 2023-07-06T01:19:08.838 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.834+0000 7f24d54c6700 1 --2- 172.21.15.103:0/3996836086 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24d00cc320 0x7f24d00b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.838 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.834+0000 7f24d54c6700 1 --2- 172.21.15.103:0/3996836086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24d006a000 0x7f24d015f4d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.839 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.834+0000 7f24d54c6700 1 --2- 172.21.15.103:0/3996836086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24d00af2a0 0x7f24d0158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:08.839 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.834+0000 7f24d54c6700 1 -- 172.21.15.103:0/3996836086 >> 172.21.15.103:0/3996836086 conn(0x7f24d00b1890 msgr2=0x7f24d006d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:08.839 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.834+0000 7f24d54c6700 1 -- 172.21.15.103:0/3996836086 shutdown_connections 2023-07-06T01:19:08.839 INFO:teuthology.orchestra.run.smithi103.stderr:2023-07-06T01:19:08.834+0000 7f24d54c6700 1 -- 172.21.15.103:0/3996836086 wait complete. 2023-07-06T01:19:08.848 INFO:tasks.ceph.osd.0:Restarting daemon 2023-07-06T01:19:08.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-07-06T01:19:08.851 INFO:tasks.ceph.osd.0:Started 2023-07-06T01:19:08.851 INFO:tasks.ceph.osd.1:Restarting daemon 2023-07-06T01:19:08.852 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-07-06T01:19:08.854 INFO:tasks.ceph.osd.1:Started 2023-07-06T01:19:08.854 INFO:tasks.ceph.osd.2:Restarting daemon 2023-07-06T01:19:08.854 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-07-06T01:19:08.856 INFO:tasks.ceph.osd.2:Started 2023-07-06T01:19:08.856 INFO:tasks.ceph.osd.3:Restarting daemon 2023-07-06T01:19:08.857 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-07-06T01:19:08.859 INFO:tasks.ceph.osd.3:Started 2023-07-06T01:19:08.859 INFO:tasks.ceph.osd.4:Restarting daemon 2023-07-06T01:19:08.859 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-07-06T01:19:08.861 INFO:tasks.ceph.osd.4:Started 2023-07-06T01:19:08.861 INFO:tasks.ceph.osd.5:Restarting daemon 2023-07-06T01:19:08.862 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-07-06T01:19:08.864 INFO:tasks.ceph.osd.5:Started 2023-07-06T01:19:08.864 INFO:tasks.ceph.osd.6:Restarting daemon 2023-07-06T01:19:08.864 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-07-06T01:19:08.866 INFO:tasks.ceph.osd.6:Started 2023-07-06T01:19:08.867 INFO:tasks.ceph.osd.7:Restarting daemon 2023-07-06T01:19:08.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-07-06T01:19:08.869 INFO:tasks.ceph.osd.7:Started 2023-07-06T01:19:08.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:08.902 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:08.901+0000 7fde73547200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T01:19:09.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.026+0000 7f6c5ddad700 1 Processor -- start 2023-07-06T01:19:09.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.026+0000 7f6c5ddad700 1 -- start start 2023-07-06T01:19:09.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c5ddad700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c5806b820 0x7f6c580cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:09.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c5ddad700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c580ce690 con 0x7f6c5815c750 2023-07-06T01:19:09.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c5ddad700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c580d06f0 con 0x7f6c580cfc30 2023-07-06T01:19:09.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c5ddad700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580ceb90 con 0x7f6c5806b820 2023-07-06T01:19:09.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c577fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6c5815c750 0x7f6c580cdf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35494/0 (socket says 172.21.15.22:35494) 2023-07-06T01:19:09.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c577fe700 1 -- 172.21.15.22:0/960911708 learned_addr learned my addr 172.21.15.22:0/960911708 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:09.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c57fff700 1 --2- 172.21.15.22:0/960911708 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c5806b820 0x7f6c580cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:09.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c567fc700 1 -- 172.21.15.22:0/960911708 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928412112 0 0) 0x7f6c580ce690 con 0x7f6c5815c750 2023-07-06T01:19:09.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c567fc700 1 -- 172.21.15.22:0/960911708 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c400035b0 con 0x7f6c5815c750 2023-07-06T01:19:09.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c567fc700 1 -- 172.21.15.22:0/960911708 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277077615 0 0) 0x7f6c580d06f0 con 0x7f6c580cfc30 2023-07-06T01:19:09.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c567fc700 1 -- 172.21.15.22:0/960911708 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c580ce690 con 0x7f6c580cfc30 2023-07-06T01:19:09.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c57fff700 1 -- 172.21.15.22:0/960911708 >> v1:172.21.15.103:6789/0 conn(0x7f6c580cfc30 legacy=0x7f6c5806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:09.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c567fc700 1 -- 172.21.15.22:0/960911708 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4188912028 0 0) 0x7f6c400035b0 con 0x7f6c5815c750 2023-07-06T01:19:09.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.027+0000 7f6c57fff700 1 -- 172.21.15.22:0/960911708 >> v1:172.21.15.22:6789/0 conn(0x7f6c5815c750 legacy=0x7f6c580cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:09.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.028+0000 7f6c57fff700 1 -- 172.21.15.22:0/960911708 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c580cee70 con 0x7f6c5806b820 2023-07-06T01:19:09.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.028+0000 7f6c57fff700 1 --2- 172.21.15.22:0/960911708 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c5806b820 0x7f6c580cb2e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009870 tx=0x7f6c50009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec1b4316333bb99 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:09.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.028+0000 7f6c567fc700 1 -- 172.21.15.22:0/960911708 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480019f0 con 0x7f6c5806b820 2023-07-06T01:19:09.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.028+0000 7f6c567fc700 1 -- 172.21.15.22:0/960911708 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c50004030 con 0x7f6c5806b820 2023-07-06T01:19:09.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.028+0000 7f6c567fc700 1 -- 172.21.15.22:0/960911708 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c500043c0 con 0x7f6c5806b820 2023-07-06T01:19:09.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.028+0000 7f6c5ddad700 1 -- 172.21.15.22:0/960911708 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c5806b820 msgr2=0x7f6c580cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:09.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.028+0000 7f6c5ddad700 1 --2- 172.21.15.22:0/960911708 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c5806b820 0x7f6c580cb2e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009870 tx=0x7f6c50009b80 comp rx=0 tx=0).stop 2023-07-06T01:19:09.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.028+0000 7f6c5ddad700 1 -- 172.21.15.22:0/960911708 shutdown_connections 2023-07-06T01:19:09.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.028+0000 7f6c5ddad700 1 --2- 172.21.15.22:0/960911708 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c5806b820 0x7f6c580cb2e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:09.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.028+0000 7f6c5ddad700 1 -- 172.21.15.22:0/960911708 >> 172.21.15.22:0/960911708 conn(0x7f6c580999f0 msgr2=0x7f6c58099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:09.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.028+0000 7f6c5ddad700 1 -- 172.21.15.22:0/960911708 shutdown_connections 2023-07-06T01:19:09.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.029+0000 7f6c5ddad700 1 -- 172.21.15.22:0/960911708 wait complete. 2023-07-06T01:19:09.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.030+0000 7f6c5ddad700 1 Processor -- start 2023-07-06T01:19:09.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.030+0000 7f6c5ddad700 1 -- start start 2023-07-06T01:19:09.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c5ddad700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c5806b820 0x7f6c58148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:09.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c5ddad700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c580cfc30 0x7f6c5815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:09.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c5ddad700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c5815c750 0x7f6c5815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:09.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c5ddad700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c581f3cb0 con 0x7f6c5815c750 2023-07-06T01:19:09.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c5ddad700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c58148670 con 0x7f6c5806b820 2023-07-06T01:19:09.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c5ddad700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c581f30e0 con 0x7f6c580cfc30 2023-07-06T01:19:09.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c57fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c5815c750 0x7f6c5815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:09.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c57fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c5815c750 0x7f6c5815afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44760/0 (socket says 172.21.15.22:44760) 2023-07-06T01:19:09.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c57fff700 1 -- 172.21.15.22:0/3456219674 learned_addr learned my addr 172.21.15.22:0/3456219674 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:09.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c577fe700 1 --2- 172.21.15.22:0/3456219674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c5806b820 0x7f6c58148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:09.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c56ffd700 1 --2- 172.21.15.22:0/3456219674 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c580cfc30 0x7f6c5815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:09.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c57fff700 1 -- 172.21.15.22:0/3456219674 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c580cfc30 msgr2=0x7f6c5815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:09.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c57fff700 1 --2- 172.21.15.22:0/3456219674 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c580cfc30 0x7f6c5815aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:09.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c57fff700 1 -- 172.21.15.22:0/3456219674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c5806b820 msgr2=0x7f6c58148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:09.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c57fff700 1 --2- 172.21.15.22:0/3456219674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c5806b820 0x7f6c58148130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:09.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c57fff700 1 -- 172.21.15.22:0/3456219674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c500094a0 con 0x7f6c5815c750 2023-07-06T01:19:09.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c57fff700 1 --2- 172.21.15.22:0/3456219674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c5815c750 0x7f6c5815afe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6c500090b0 tx=0x7f6c50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:09.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.031+0000 7f6c3ffff700 1 -- 172.21.15.22:0/3456219674 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50003cf0 con 0x7f6c5815c750 2023-07-06T01:19:09.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.032+0000 7f6c3ffff700 1 -- 172.21.15.22:0/3456219674 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5001f070 con 0x7f6c5815c750 2023-07-06T01:19:09.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.032+0000 7f6c3ffff700 1 -- 172.21.15.22:0/3456219674 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50011870 con 0x7f6c5815c750 2023-07-06T01:19:09.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.032+0000 7f6c5ddad700 1 -- 172.21.15.22:0/3456219674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c581f3380 con 0x7f6c5815c750 2023-07-06T01:19:09.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.032+0000 7f6c5ddad700 1 -- 172.21.15.22:0/3456219674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c581f3940 con 0x7f6c5815c750 2023-07-06T01:19:09.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.032+0000 7f6c3ffff700 1 -- 172.21.15.22:0/3456219674 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6c50011cd0 con 0x7f6c5815c750 2023-07-06T01:19:09.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.032+0000 7f6c3ffff700 1 -- 172.21.15.22:0/3456219674 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f6c50021070 con 0x7f6c5815c750 2023-07-06T01:19:09.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.033+0000 7f6c5ddad700 1 -- 172.21.15.22:0/3456219674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c581f3e20 con 0x7f6c5815c750 2023-07-06T01:19:09.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.035+0000 7f6c3ffff700 1 -- 172.21.15.22:0/3456219674 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f6c5002b020 con 0x7f6c5815c750 2023-07-06T01:19:09.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.106+0000 7f6c5ddad700 1 -- 172.21.15.22:0/3456219674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c580744b0 con 0x7f6c5815c750 2023-07-06T01:19:09.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.106+0000 7f6c3ffff700 1 -- 172.21.15.22:0/3456219674 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f6c50026c60 con 0x7f6c5815c750 2023-07-06T01:19:09.108 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:09.109 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":10,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:08.827237+0000","last_up_change":"0.000000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:09.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.107+0000 7f6c5ddad700 1 -- 172.21.15.22:0/3456219674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c5815c750 msgr2=0x7f6c5815afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.107+0000 7f6c5ddad700 1 --2- 172.21.15.22:0/3456219674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c5815c750 0x7f6c5815afe0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6c500090b0 tx=0x7f6c50004750 comp rx=0 tx=0).stop 2023-07-06T01:19:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.107+0000 7f6c5ddad700 1 -- 172.21.15.22:0/3456219674 shutdown_connections 2023-07-06T01:19:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.107+0000 7f6c5ddad700 1 --2- 172.21.15.22:0/3456219674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c5815c750 0x7f6c5815afe0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.107+0000 7f6c5ddad700 1 --2- 172.21.15.22:0/3456219674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c5806b820 0x7f6c58148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.107+0000 7f6c5ddad700 1 --2- 172.21.15.22:0/3456219674 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c580cfc30 0x7f6c5815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.107+0000 7f6c5ddad700 1 -- 172.21.15.22:0/3456219674 >> 172.21.15.22:0/3456219674 conn(0x7f6c580999f0 msgr2=0x7f6c580ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:09.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.107+0000 7f6c5ddad700 1 -- 172.21.15.22:0/3456219674 shutdown_connections 2023-07-06T01:19:09.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:09.107+0000 7f6c5ddad700 1 -- 172.21.15.22:0/3456219674 wait complete. 2023-07-06T01:19:09.121 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-07-06T01:19:09.122 INFO:tasks.ceph:Waiting for OSDs to come up 2023-07-06T01:19:09.461 INFO:tasks.ceph.osd.5.smithi103.stderr:2023-07-06T01:19:09.459+0000 7faacab4f640 -1 Falling back to public interface 2023-07-06T01:19:09.477 INFO:tasks.ceph.osd.6.smithi103.stderr:2023-07-06T01:19:09.475+0000 7f487d517640 -1 Falling back to public interface 2023-07-06T01:19:09.485 INFO:tasks.ceph.osd.7.smithi103.stderr:2023-07-06T01:19:09.483+0000 7f2714ade640 -1 Falling back to public interface 2023-07-06T01:19:09.486 INFO:tasks.ceph.osd.1.smithi022.stderr:2023-07-06T01:19:09.484+0000 7f9d2a5c0640 -1 Falling back to public interface 2023-07-06T01:19:09.489 INFO:tasks.ceph.osd.0.smithi022.stderr:2023-07-06T01:19:09.487+0000 7f085a65f640 -1 Falling back to public interface 2023-07-06T01:19:09.490 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:09.489+0000 7f786f557200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T01:19:09.493 INFO:tasks.ceph.osd.2.smithi022.stderr:2023-07-06T01:19:09.491+0000 7fd617b63640 -1 Falling back to public interface 2023-07-06T01:19:09.494 INFO:tasks.ceph.osd.4.smithi103.stderr:2023-07-06T01:19:09.492+0000 7f9e42db7640 -1 Falling back to public interface 2023-07-06T01:19:09.496 INFO:tasks.ceph.osd.3.smithi022.stderr:2023-07-06T01:19:09.495+0000 7f33008ff640 -1 Falling back to public interface 2023-07-06T01:19:09.569 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:09.567+0000 7f786f557200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T01:19:09.658 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:09.656+0000 7fde73547200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T01:19:09.708 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:09.706+0000 7f786f557200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T01:19:09.738 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:09.737+0000 7fde73547200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T01:19:09.885 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:09.883+0000 7fde73547200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T01:19:09.930 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:09.943 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:09.941+0000 7f786f557200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T01:19:10.016 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:10.015+0000 7f786f557200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T01:19:10.045 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.043+0000 7f16c88da700 1 Processor -- start 2023-07-06T01:19:10.045 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.044+0000 7f16c88da700 1 -- start start 2023-07-06T01:19:10.047 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c88da700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16c415a840 0x7f16c4071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:10.047 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c88da700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f16c4071b80 0x7f16c406a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:10.047 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c88da700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c40cb7b0 con 0x7f16c406d240 2023-07-06T01:19:10.047 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c88da700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c40cbb50 con 0x7f16c4071b80 2023-07-06T01:19:10.047 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c88da700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c406d6d0 con 0x7f16c415a840 2023-07-06T01:19:10.047 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c1d9b700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f16c4071b80 0x7f16c406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:10.048 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c2d9d700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f16c406d240 0x7f16c415e420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:37574/0 (socket says 172.21.15.22:37574) 2023-07-06T01:19:10.048 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c2d9d700 1 -- 172.21.15.22:0/2374568169 learned_addr learned my addr 172.21.15.22:0/2374568169 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:10.048 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c259c700 1 --2- 172.21.15.22:0/2374568169 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16c415a840 0x7f16c4071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:10.049 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c259c700 1 -- 172.21.15.22:0/2374568169 >> v2:172.21.15.103:3300/0 conn(0x7f16c4071b80 msgr2=0x7f16c406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:10.049 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c259c700 1 --2- 172.21.15.22:0/2374568169 >> v2:172.21.15.103:3300/0 conn(0x7f16c4071b80 0x7f16c406a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:10.049 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c259c700 1 -- 172.21.15.22:0/2374568169 >> v1:172.21.15.103:6789/0 conn(0x7f16c406d240 legacy=0x7f16c415e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:10.050 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c159a700 1 -- 172.21.15.22:0/2374568169 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192362615 0 0) 0x7f16c40cb7b0 con 0x7f16c406d240 2023-07-06T01:19:10.050 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.045+0000 7f16c259c700 1 -- 172.21.15.22:0/2374568169 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c406d9b0 con 0x7f16c415a840 2023-07-06T01:19:10.050 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.046+0000 7f16c1d9b700 1 --2- 172.21.15.22:0/2374568169 >> v2:172.21.15.103:3300/0 conn(0x7f16c4071b80 0x7f16c406a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:10.050 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.046+0000 7f16c259c700 1 --2- 172.21.15.22:0/2374568169 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16c415a840 0x7f16c4071640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009840 tx=0x7f16b4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b3cc4e4d8547390 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:10.050 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.046+0000 7f16c159a700 1 -- 172.21.15.22:0/2374568169 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4011830 con 0x7f16c415a840 2023-07-06T01:19:10.051 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.046+0000 7f16c159a700 1 -- 172.21.15.22:0/2374568169 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b4011990 con 0x7f16c415a840 2023-07-06T01:19:10.051 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.046+0000 7f16c159a700 1 -- 172.21.15.22:0/2374568169 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4011d20 con 0x7f16c415a840 2023-07-06T01:19:10.051 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.046+0000 7f16c88da700 1 -- 172.21.15.22:0/2374568169 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16c415a840 msgr2=0x7f16c4071640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:10.051 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.046+0000 7f16c88da700 1 --2- 172.21.15.22:0/2374568169 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16c415a840 0x7f16c4071640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009840 tx=0x7f16b4009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:10.051 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.046+0000 7f16c88da700 1 -- 172.21.15.22:0/2374568169 shutdown_connections 2023-07-06T01:19:10.051 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.046+0000 7f16c88da700 1 --2- 172.21.15.22:0/2374568169 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16c415a840 0x7f16c4071640 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:10.051 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.046+0000 7f16c88da700 1 --2- 172.21.15.22:0/2374568169 >> v2:172.21.15.103:3300/0 conn(0x7f16c4071b80 0x7f16c406a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:10.052 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.046+0000 7f16c88da700 1 -- 172.21.15.22:0/2374568169 >> 172.21.15.22:0/2374568169 conn(0x7f16c40b19a0 msgr2=0x7f16c40b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:10.052 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.047+0000 7f16c88da700 1 -- 172.21.15.22:0/2374568169 shutdown_connections 2023-07-06T01:19:10.052 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.047+0000 7f16c88da700 1 -- 172.21.15.22:0/2374568169 wait complete. 2023-07-06T01:19:10.054 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.048+0000 7f16c88da700 1 Processor -- start 2023-07-06T01:19:10.054 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.048+0000 7f16c88da700 1 -- start start 2023-07-06T01:19:10.054 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c88da700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16c406d240 0x7f16c41f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:10.054 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c88da700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4071b80 0x7f16c40b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:10.054 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c88da700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c415a840 0x7f16c41ef180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:10.054 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c88da700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40c7ed0 con 0x7f16c4071b80 2023-07-06T01:19:10.054 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c88da700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40c8490 con 0x7f16c415a840 2023-07-06T01:19:10.055 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c88da700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40b58c0 con 0x7f16c406d240 2023-07-06T01:19:10.055 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c1d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4071b80 0x7f16c40b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:10.055 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c1d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4071b80 0x7f16c40b2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44770/0 (socket says 172.21.15.22:44770) 2023-07-06T01:19:10.055 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c1d9b700 1 -- 172.21.15.22:0/1818277098 learned_addr learned my addr 172.21.15.22:0/1818277098 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:10.055 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c259c700 1 --2- 172.21.15.22:0/1818277098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16c406d240 0x7f16c41f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:10.056 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c259c700 1 -- 172.21.15.22:0/1818277098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c415a840 msgr2=0x7f16c41ef180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T01:19:10.056 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c259c700 1 --2- 172.21.15.22:0/1818277098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c415a840 0x7f16c41ef180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:10.056 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c259c700 1 -- 172.21.15.22:0/1818277098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4071b80 msgr2=0x7f16c40b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:10.056 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c259c700 1 --2- 172.21.15.22:0/1818277098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4071b80 0x7f16c40b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:10.056 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c259c700 1 -- 172.21.15.22:0/1818277098 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b80094a0 con 0x7f16c406d240 2023-07-06T01:19:10.056 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c1d9b700 1 --2- 172.21.15.22:0/1818277098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4071b80 0x7f16c40b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:10.056 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16c259c700 1 --2- 172.21.15.22:0/1818277098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16c406d240 0x7f16c41f3580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f16b4016000 tx=0x7f16b4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:10.056 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.050+0000 7f16b2ffd700 1 -- 172.21.15.22:0/1818277098 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b40036a0 con 0x7f16c406d240 2023-07-06T01:19:10.057 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.051+0000 7f16b2ffd700 1 -- 172.21.15.22:0/1818277098 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b401e070 con 0x7f16c406d240 2023-07-06T01:19:10.057 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.051+0000 7f16b2ffd700 1 -- 172.21.15.22:0/1818277098 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4003a20 con 0x7f16c406d240 2023-07-06T01:19:10.057 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.051+0000 7f16c88da700 1 -- 172.21.15.22:0/1818277098 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b40094a0 con 0x7f16c406d240 2023-07-06T01:19:10.057 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.051+0000 7f16c88da700 1 -- 172.21.15.22:0/1818277098 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c415ad90 con 0x7f16c406d240 2023-07-06T01:19:10.058 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.051+0000 7f16b2ffd700 1 -- 172.21.15.22:0/1818277098 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f16b401a420 con 0x7f16c406d240 2023-07-06T01:19:10.058 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.051+0000 7f16b2ffd700 1 -- 172.21.15.22:0/1818277098 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f16b4024ad0 con 0x7f16c406d240 2023-07-06T01:19:10.058 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.052+0000 7f16c88da700 1 -- 172.21.15.22:0/1818277098 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c40b5db0 con 0x7f16c406d240 2023-07-06T01:19:10.059 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.054+0000 7f16b2ffd700 1 -- 172.21.15.22:0/1818277098 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f16b4020070 con 0x7f16c406d240 2023-07-06T01:19:10.126 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.124+0000 7f16c88da700 1 -- 172.21.15.22:0/1818277098 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16c40744e0 con 0x7f16c406d240 2023-07-06T01:19:10.127 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.125+0000 7f16b2ffd700 1 -- 172.21.15.22:0/1818277098 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f16b4015020 con 0x7f16c406d240 2023-07-06T01:19:10.127 INFO:teuthology.misc.health.smithi022.stdout: 2023-07-06T01:19:10.127 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":10,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:08.827237+0000","last_up_change":"0.000000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:10.128 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.126+0000 7f16c88da700 1 -- 172.21.15.22:0/1818277098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16c406d240 msgr2=0x7f16c41f3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:10.128 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.126+0000 7f16c88da700 1 --2- 172.21.15.22:0/1818277098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16c406d240 0x7f16c41f3580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f16b4016000 tx=0x7f16b4004750 comp rx=0 tx=0).stop 2023-07-06T01:19:10.129 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.126+0000 7f16c88da700 1 -- 172.21.15.22:0/1818277098 shutdown_connections 2023-07-06T01:19:10.129 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.126+0000 7f16c88da700 1 --2- 172.21.15.22:0/1818277098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16c415a840 0x7f16c41ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:10.129 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.126+0000 7f16c88da700 1 --2- 172.21.15.22:0/1818277098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16c406d240 0x7f16c41f3580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:10.130 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.126+0000 7f16c88da700 1 --2- 172.21.15.22:0/1818277098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16c4071b80 0x7f16c40b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:10.130 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.126+0000 7f16c88da700 1 -- 172.21.15.22:0/1818277098 >> 172.21.15.22:0/1818277098 conn(0x7f16c40b19a0 msgr2=0x7f16c40af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:10.130 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.126+0000 7f16c88da700 1 -- 172.21.15.22:0/1818277098 shutdown_connections 2023-07-06T01:19:10.130 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:10.126+0000 7f16c88da700 1 -- 172.21.15.22:0/1818277098 wait complete. 2023-07-06T01:19:10.133 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:10.132+0000 7fde73547200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T01:19:10.139 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-06T01:19:10.212 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:10.211+0000 7fde73547200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T01:19:10.287 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:10.286+0000 7f786f557200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T01:19:10.500 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:10.498+0000 7fde73547200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T01:19:10.504 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:10.502+0000 7f786f557200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T01:19:10.660 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:10.657+0000 7f786f557200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T01:19:10.728 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:10.726+0000 7fde73547200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T01:19:10.775 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:10.772+0000 7f786f557200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T01:19:10.891 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:10.889+0000 7fde73547200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T01:19:10.923 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:10.921+0000 7f786f557200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T01:19:11.010 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:11.009+0000 7fde73547200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T01:19:11.166 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:11.165+0000 7fde73547200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T01:19:11.198 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:11.195+0000 7f786f557200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T01:19:11.382 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:11.380+0000 7f786f557200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T01:19:11.452 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:11.451+0000 7fde73547200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T01:19:11.645 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:11.643+0000 7fde73547200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T01:19:11.978 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:11.975+0000 7f786f557200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T01:19:12.289 INFO:tasks.ceph.mgr.y.smithi022.stderr:2023-07-06T01:19:12.287+0000 7fde73547200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T01:19:12.965 INFO:tasks.ceph.osd.5.smithi103.stderr:2023-07-06T01:19:12.962+0000 7faacab4f640 -1 osd.5 0 log_to_monitors true 2023-07-06T01:19:13.000 INFO:tasks.ceph.osd.6.smithi103.stderr:2023-07-06T01:19:12.997+0000 7f487d517640 -1 osd.6 0 log_to_monitors true 2023-07-06T01:19:13.007 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:13.004+0000 7f78277be700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-06T01:19:13.007 INFO:tasks.ceph.osd.4.smithi103.stderr:2023-07-06T01:19:13.004+0000 7f9e42db7640 -1 osd.4 0 log_to_monitors true 2023-07-06T01:19:13.008 INFO:tasks.ceph.mgr.x.smithi103.stderr:2023-07-06T01:19:13.004+0000 7f78277be700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-06T01:19:13.030 INFO:tasks.ceph.osd.7.smithi103.stderr:2023-07-06T01:19:13.028+0000 7f2714ade640 -1 osd.7 0 log_to_monitors true 2023-07-06T01:19:13.040 INFO:tasks.ceph.osd.3.smithi022.stderr:2023-07-06T01:19:13.038+0000 7f33008ff640 -1 osd.3 0 log_to_monitors true 2023-07-06T01:19:13.041 INFO:tasks.ceph.osd.2.smithi022.stderr:2023-07-06T01:19:13.039+0000 7fd617b63640 -1 osd.2 0 log_to_monitors true 2023-07-06T01:19:13.046 INFO:tasks.ceph.osd.0.smithi022.stderr:2023-07-06T01:19:13.043+0000 7f085a65f640 -1 osd.0 0 log_to_monitors true 2023-07-06T01:19:13.046 INFO:tasks.ceph.osd.1.smithi022.stderr:2023-07-06T01:19:13.044+0000 7f9d2a5c0640 -1 osd.1 0 log_to_monitors true 2023-07-06T01:19:15.464 INFO:tasks.ceph.osd.3.smithi022.stderr:2023-07-06T01:19:15.462+0000 7f32f44b6700 -1 osd.3 0 waiting for initial osdmap 2023-07-06T01:19:15.471 INFO:tasks.ceph.osd.1.smithi022.stderr:2023-07-06T01:19:15.469+0000 7f9d1e177700 -1 osd.1 0 waiting for initial osdmap 2023-07-06T01:19:15.479 INFO:tasks.ceph.osd.3.smithi022.stderr:2023-07-06T01:19:15.478+0000 7f32ec891700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T01:19:15.487 INFO:tasks.ceph.osd.1.smithi022.stderr:2023-07-06T01:19:15.486+0000 7f9d16552700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T01:19:15.514 INFO:tasks.ceph.osd.0.smithi022.stderr:2023-07-06T01:19:15.512+0000 7f084e216700 -1 osd.0 0 waiting for initial osdmap 2023-07-06T01:19:15.524 INFO:tasks.ceph.osd.0.smithi022.stderr:2023-07-06T01:19:15.523+0000 7f08465f1700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T01:19:15.538 INFO:tasks.ceph.osd.2.smithi022.stderr:2023-07-06T01:19:15.536+0000 7fd60b71a700 -1 osd.2 0 waiting for initial osdmap 2023-07-06T01:19:15.548 INFO:tasks.ceph.osd.2.smithi022.stderr:2023-07-06T01:19:15.546+0000 7fd603af5700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T01:19:16.159 INFO:tasks.ceph.osd.6.smithi103.stderr:2023-07-06T01:19:16.154+0000 7f48710ce700 -1 osd.6 0 waiting for initial osdmap 2023-07-06T01:19:16.174 INFO:tasks.ceph.osd.6.smithi103.stderr:2023-07-06T01:19:16.171+0000 7f48694a9700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T01:19:16.194 INFO:tasks.ceph.osd.5.smithi103.stderr:2023-07-06T01:19:16.191+0000 7faabe706700 -1 osd.5 0 waiting for initial osdmap 2023-07-06T01:19:16.208 INFO:tasks.ceph.osd.5.smithi103.stderr:2023-07-06T01:19:16.206+0000 7faab6ae1700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T01:19:16.212 INFO:tasks.ceph.osd.4.smithi103.stderr:2023-07-06T01:19:16.210+0000 7f9e3696e700 -1 osd.4 0 waiting for initial osdmap 2023-07-06T01:19:16.213 INFO:tasks.ceph.osd.7.smithi103.stderr:2023-07-06T01:19:16.210+0000 7f2708695700 -1 osd.7 0 waiting for initial osdmap 2023-07-06T01:19:16.230 INFO:tasks.ceph.osd.7.smithi103.stderr:2023-07-06T01:19:16.228+0000 7f2700a70700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T01:19:16.232 INFO:tasks.ceph.osd.4.smithi103.stderr:2023-07-06T01:19:16.230+0000 7f9e2ed49700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T01:19:16.946 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:17.075 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.073+0000 7f8b7ab2d700 1 Processor -- start 2023-07-06T01:19:17.075 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.073+0000 7f8b7ab2d700 1 -- start start 2023-07-06T01:19:17.077 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.074+0000 7f8b7ab2d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8b740ce6d0 0x7f8b7406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:17.077 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.074+0000 7f8b7ab2d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b7415d310 0x7f8b740caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:17.078 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.074+0000 7f8b7ab2d700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7406deb0 con 0x7f8b741597a0 2023-07-06T01:19:17.078 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.074+0000 7f8b7ab2d700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b740b2950 con 0x7f8b740ce6d0 2023-07-06T01:19:17.078 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.074+0000 7f8b7ab2d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7406d090 con 0x7f8b7415d310 2023-07-06T01:19:17.078 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.074+0000 7f8b73fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8b740ce6d0 0x7f8b7406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:17.078 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.074+0000 7f8b73fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8b740ce6d0 0x7f8b7406cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44922/0 (socket says 172.21.15.22:44922) 2023-07-06T01:19:17.078 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.074+0000 7f8b73fff700 1 -- 172.21.15.22:0/1508998177 learned_addr learned my addr 172.21.15.22:0/1508998177 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:17.079 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.075+0000 7f8b790ca700 1 --2- 172.21.15.22:0/1508998177 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b7415d310 0x7f8b740caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:17.079 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.075+0000 7f8b73fff700 1 -- 172.21.15.22:0/1508998177 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b7415d310 msgr2=0x7f8b740caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:17.079 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.075+0000 7f8b73fff700 1 --2- 172.21.15.22:0/1508998177 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b7415d310 0x7f8b740caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.079 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.075+0000 7f8b73fff700 1 -- 172.21.15.22:0/1508998177 >> v1:172.21.15.103:6789/0 conn(0x7f8b741597a0 legacy=0x7f8b740cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:17.079 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.075+0000 7f8b73fff700 1 -- 172.21.15.22:0/1508998177 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7406d2a0 con 0x7f8b740ce6d0 2023-07-06T01:19:17.079 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.075+0000 7f8b73fff700 1 --2- 172.21.15.22:0/1508998177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b740ce6d0 0x7f8b7406cb50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8b600098f0 tx=0x7f8b60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b0881f6eb03407 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:17.079 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.075+0000 7f8b737fe700 1 -- 172.21.15.22:0/1508998177 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000f8a0 con 0x7f8b740ce6d0 2023-07-06T01:19:17.080 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.075+0000 7f8b737fe700 1 -- 172.21.15.22:0/1508998177 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6000fa00 con 0x7f8b740ce6d0 2023-07-06T01:19:17.080 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.075+0000 7f8b737fe700 1 -- 172.21.15.22:0/1508998177 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000fd90 con 0x7f8b740ce6d0 2023-07-06T01:19:17.080 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.076+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/1508998177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b740ce6d0 msgr2=0x7f8b7406cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:17.080 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.076+0000 7f8b7ab2d700 1 --2- 172.21.15.22:0/1508998177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b740ce6d0 0x7f8b7406cb50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8b600098f0 tx=0x7f8b60009c00 comp rx=0 tx=0).stop 2023-07-06T01:19:17.080 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.076+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/1508998177 shutdown_connections 2023-07-06T01:19:17.080 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.076+0000 7f8b7ab2d700 1 --2- 172.21.15.22:0/1508998177 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b7415d310 0x7f8b740caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.080 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.076+0000 7f8b7ab2d700 1 --2- 172.21.15.22:0/1508998177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b740ce6d0 0x7f8b7406cb50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.081 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.076+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/1508998177 >> 172.21.15.22:0/1508998177 conn(0x7f8b740b19a0 msgr2=0x7f8b740b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:17.081 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.076+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/1508998177 shutdown_connections 2023-07-06T01:19:17.081 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.076+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/1508998177 wait complete. 2023-07-06T01:19:17.083 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.077+0000 7f8b7ab2d700 1 Processor -- start 2023-07-06T01:19:17.083 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.077+0000 7f8b7ab2d700 1 -- start start 2023-07-06T01:19:17.083 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b7ab2d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b740ce6d0 0x7f8b740c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:17.084 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b7ab2d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b741597a0 0x7f8b740c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:17.084 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b7ab2d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b7415d310 0x7f8b740b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:17.084 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b7ab2d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b740c3480 con 0x7f8b740ce6d0 2023-07-06T01:19:17.084 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b7ab2d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b740c9d80 con 0x7f8b741597a0 2023-07-06T01:19:17.084 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b7ab2d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b740c36d0 con 0x7f8b7415d310 2023-07-06T01:19:17.084 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b788c9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b740ce6d0 0x7f8b740c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:17.085 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b788c9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b740ce6d0 0x7f8b740c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44928/0 (socket says 172.21.15.22:44928) 2023-07-06T01:19:17.085 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b788c9700 1 -- 172.21.15.22:0/4226671862 learned_addr learned my addr 172.21.15.22:0/4226671862 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:17.085 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b790ca700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b7415d310 0x7f8b740b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:17.085 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b73fff700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b741597a0 0x7f8b740c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:17.085 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b788c9700 1 -- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b7415d310 msgr2=0x7f8b740b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:17.085 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b788c9700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b7415d310 0x7f8b740b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.085 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b788c9700 1 -- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b741597a0 msgr2=0x7f8b740c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:17.086 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b788c9700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b741597a0 0x7f8b740c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.086 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b788c9700 1 -- 172.21.15.22:0/4226671862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b60009550 con 0x7f8b740ce6d0 2023-07-06T01:19:17.086 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b73fff700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b741597a0 0x7f8b740c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:17.086 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.078+0000 7f8b790ca700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b7415d310 0x7f8b740b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:17.086 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.079+0000 7f8b788c9700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b740ce6d0 0x7f8b740c7fe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800c4e0 tx=0x7f8b6800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:17.086 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.079+0000 7f8b717fa700 1 -- 172.21.15.22:0/4226671862 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b680052d0 con 0x7f8b740ce6d0 2023-07-06T01:19:17.086 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.079+0000 7f8b717fa700 1 -- 172.21.15.22:0/4226671862 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b68005430 con 0x7f8b740ce6d0 2023-07-06T01:19:17.087 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.079+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/4226671862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b740c39b0 con 0x7f8b740ce6d0 2023-07-06T01:19:17.087 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.079+0000 7f8b717fa700 1 -- 172.21.15.22:0/4226671862 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b680038c0 con 0x7f8b740ce6d0 2023-07-06T01:19:17.087 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.079+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/4226671862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b740c3f70 con 0x7f8b740ce6d0 2023-07-06T01:19:17.087 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.080+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/4226671862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b74066ad0 con 0x7f8b740ce6d0 2023-07-06T01:19:17.087 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.080+0000 7f8b717fa700 1 -- 172.21.15.22:0/4226671862 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f8b68003a20 con 0x7f8b740ce6d0 2023-07-06T01:19:17.087 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.081+0000 7f8b717fa700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8b5806ac50 0x7f8b5806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:17.087 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.081+0000 7f8b717fa700 1 -- 172.21.15.22:0/4226671862 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3911+0+0 (secure 0 0 0) 0x7f8b680880d0 con 0x7f8b740ce6d0 2023-07-06T01:19:17.088 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.081+0000 7f8b73fff700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8b5806ac50 0x7f8b5806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:17.088 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.081+0000 7f8b73fff700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8b5806ac50 0x7f8b5806d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8b600098f0 tx=0x7f8b60008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:17.088 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.085+0000 7f8b717fa700 1 -- 172.21.15.22:0/4226671862 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8b680ba310 con 0x7f8b740ce6d0 2023-07-06T01:19:17.290 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.288+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/4226671862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b740744e0 con 0x7f8b740ce6d0 2023-07-06T01:19:17.290 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.289+0000 7f8b717fa700 1 -- 172.21.15.22:0/4226671862 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+11034 (secure 0 0 0) 0x7f8b68017090 con 0x7f8b740ce6d0 2023-07-06T01:19:17.291 INFO:teuthology.misc.health.smithi022.stdout: 2023-07-06T01:19:17.291 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":14,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:17.235081+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:17.293 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.291+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8b5806ac50 msgr2=0x7f8b5806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:17.293 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.291+0000 7f8b7ab2d700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8b5806ac50 0x7f8b5806d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8b600098f0 tx=0x7f8b60008030 comp rx=0 tx=0).stop 2023-07-06T01:19:17.293 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.291+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/4226671862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b740ce6d0 msgr2=0x7f8b740c7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:17.293 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.291+0000 7f8b7ab2d700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b740ce6d0 0x7f8b740c7fe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800c4e0 tx=0x7f8b6800c8a0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.293 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.291+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/4226671862 shutdown_connections 2023-07-06T01:19:17.294 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.291+0000 7f8b7ab2d700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b7415d310 0x7f8b740b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.294 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.291+0000 7f8b7ab2d700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b740ce6d0 0x7f8b740c7fe0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.294 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.291+0000 7f8b7ab2d700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8b5806ac50 0x7f8b5806d100 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.294 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.291+0000 7f8b7ab2d700 1 --2- 172.21.15.22:0/4226671862 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b741597a0 0x7f8b740c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.294 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.291+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/4226671862 >> 172.21.15.22:0/4226671862 conn(0x7f8b740b19a0 msgr2=0x7f8b740cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:17.294 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.291+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/4226671862 shutdown_connections 2023-07-06T01:19:17.294 INFO:teuthology.misc.health.smithi022.stderr:2023-07-06T01:19:17.291+0000 7f8b7ab2d700 1 -- 172.21.15.22:0/4226671862 wait complete. 2023-07-06T01:19:17.304 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-07-06T01:19:17.304 INFO:tasks.ceph:Creating RBD pool 2023-07-06T01:19:17.304 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd pool create rbd 8 2023-07-06T01:19:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.432+0000 7fee3e75e700 1 Processor -- start 2023-07-06T01:19:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.433+0000 7fee3e75e700 1 -- start start 2023-07-06T01:19:17.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.433+0000 7fee3e75e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee3815a690 0x7fee380ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:17.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.433+0000 7fee3e75e700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fee380cee60 0x7fee3806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:17.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.433+0000 7fee3e75e700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee380cd640 con 0x7fee380c9a40 2023-07-06T01:19:17.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.433+0000 7fee3e75e700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee380cd920 con 0x7fee380cee60 2023-07-06T01:19:17.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.433+0000 7fee3e75e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee380cdc00 con 0x7fee3815a690 2023-07-06T01:19:17.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee377fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee3815a690 0x7fee380ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:17.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee37fff700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fee380c9a40 0x7fee380c9e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:37766/0 (socket says 172.21.15.22:37766) 2023-07-06T01:19:17.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee3ccfb700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fee380cee60 0x7fee3806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:17.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee37fff700 1 -- 172.21.15.22:0/1026609570 learned_addr learned my addr 172.21.15.22:0/1026609570 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:17.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee36ffd700 1 -- 172.21.15.22:0/1026609570 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409455420 0 0) 0x7fee380cd640 con 0x7fee380c9a40 2023-07-06T01:19:17.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee36ffd700 1 -- 172.21.15.22:0/1026609570 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee240035b0 con 0x7fee380c9a40 2023-07-06T01:19:17.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee377fe700 1 -- 172.21.15.22:0/1026609570 >> v2:172.21.15.103:3300/0 conn(0x7fee380cee60 msgr2=0x7fee3806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:17.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee377fe700 1 --2- 172.21.15.22:0/1026609570 >> v2:172.21.15.103:3300/0 conn(0x7fee380cee60 0x7fee3806a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee377fe700 1 -- 172.21.15.22:0/1026609570 >> v1:172.21.15.103:6789/0 conn(0x7fee380c9a40 legacy=0x7fee380c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:17.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee377fe700 1 -- 172.21.15.22:0/1026609570 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee380cdee0 con 0x7fee3815a690 2023-07-06T01:19:17.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee3ccfb700 1 --2- 172.21.15.22:0/1026609570 >> v2:172.21.15.103:3300/0 conn(0x7fee380cee60 0x7fee3806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:17.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee377fe700 1 --2- 172.21.15.22:0/1026609570 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee3815a690 0x7fee380ce920 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee20009840 tx=0x7fee20009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1cd146516bafbe3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:17.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee36ffd700 1 -- 172.21.15.22:0/1026609570 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20011830 con 0x7fee3815a690 2023-07-06T01:19:17.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee36ffd700 1 -- 172.21.15.22:0/1026609570 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee20011990 con 0x7fee3815a690 2023-07-06T01:19:17.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.434+0000 7fee36ffd700 1 -- 172.21.15.22:0/1026609570 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20011d20 con 0x7fee3815a690 2023-07-06T01:19:17.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.435+0000 7fee3e75e700 1 -- 172.21.15.22:0/1026609570 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee3815a690 msgr2=0x7fee380ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:17.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.435+0000 7fee3e75e700 1 --2- 172.21.15.22:0/1026609570 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee3815a690 0x7fee380ce920 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee20009840 tx=0x7fee20009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:17.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.435+0000 7fee3e75e700 1 -- 172.21.15.22:0/1026609570 shutdown_connections 2023-07-06T01:19:17.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.435+0000 7fee3e75e700 1 --2- 172.21.15.22:0/1026609570 >> v2:172.21.15.103:3300/0 conn(0x7fee380cee60 0x7fee3806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.435+0000 7fee3e75e700 1 --2- 172.21.15.22:0/1026609570 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee3815a690 0x7fee380ce920 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.435+0000 7fee3e75e700 1 -- 172.21.15.22:0/1026609570 >> 172.21.15.22:0/1026609570 conn(0x7fee380ccb00 msgr2=0x7fee380cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.435+0000 7fee3e75e700 1 -- 172.21.15.22:0/1026609570 shutdown_connections 2023-07-06T01:19:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.435+0000 7fee3e75e700 1 -- 172.21.15.22:0/1026609570 wait complete. 2023-07-06T01:19:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.436+0000 7fee3e75e700 1 Processor -- start 2023-07-06T01:19:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.436+0000 7fee3e75e700 1 -- start start 2023-07-06T01:19:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.437+0000 7fee3e75e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee380c9a40 0x7fee3814f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.437+0000 7fee3e75e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee380cee60 0x7fee3815bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.437+0000 7fee3e75e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee3815a690 0x7fee38155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:17.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.437+0000 7fee3e75e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3815c8f0 con 0x7fee3815a690 2023-07-06T01:19:17.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.437+0000 7fee3e75e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3815cbd0 con 0x7fee380cee60 2023-07-06T01:19:17.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.437+0000 7fee3e75e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3815ceb0 con 0x7fee380c9a40 2023-07-06T01:19:17.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.437+0000 7fee3ccfb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee3815a690 0x7fee38155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:17.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.437+0000 7fee3ccfb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee3815a690 0x7fee38155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44968/0 (socket says 172.21.15.22:44968) 2023-07-06T01:19:17.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.437+0000 7fee3ccfb700 1 -- 172.21.15.22:0/3525177201 learned_addr learned my addr 172.21.15.22:0/3525177201 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:17.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.437+0000 7fee37fff700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee380c9a40 0x7fee3814f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:17.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.437+0000 7fee377fe700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee380cee60 0x7fee3815bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:17.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.438+0000 7fee3ccfb700 1 -- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee380c9a40 msgr2=0x7fee3814f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:17.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.438+0000 7fee3ccfb700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee380c9a40 0x7fee3814f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.438+0000 7fee3ccfb700 1 -- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee380cee60 msgr2=0x7fee3815bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:17.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.438+0000 7fee3ccfb700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee380cee60 0x7fee3815bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:17.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.438+0000 7fee3ccfb700 1 -- 172.21.15.22:0/3525177201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee30009a70 con 0x7fee3815a690 2023-07-06T01:19:17.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.438+0000 7fee377fe700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee380cee60 0x7fee3815bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:17.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.438+0000 7fee3ccfb700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee3815a690 0x7fee38155790 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee3000efc0 tx=0x7fee3000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:17.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.438+0000 7fee34ff9700 1 -- 172.21.15.22:0/3525177201 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30009d70 con 0x7fee3815a690 2023-07-06T01:19:17.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.438+0000 7fee34ff9700 1 -- 172.21.15.22:0/3525177201 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3000f040 con 0x7fee3815a690 2023-07-06T01:19:17.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.438+0000 7fee3e75e700 1 -- 172.21.15.22:0/3525177201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee200094a0 con 0x7fee3815a690 2023-07-06T01:19:17.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.438+0000 7fee34ff9700 1 -- 172.21.15.22:0/3525177201 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3000ce00 con 0x7fee3815a690 2023-07-06T01:19:17.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.438+0000 7fee3e75e700 1 -- 172.21.15.22:0/3525177201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3815d470 con 0x7fee3815a690 2023-07-06T01:19:17.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.439+0000 7fee3e75e700 1 -- 172.21.15.22:0/3525177201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3815d690 con 0x7fee3815a690 2023-07-06T01:19:17.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.439+0000 7fee34ff9700 1 -- 172.21.15.22:0/3525177201 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fee30014480 con 0x7fee3815a690 2023-07-06T01:19:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.440+0000 7fee34ff9700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee1806ac50 0x7fee1806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.440+0000 7fee34ff9700 1 -- 172.21.15.22:0/3525177201 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fee30089460 con 0x7fee3815a690 2023-07-06T01:19:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.440+0000 7fee37fff700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee1806ac50 0x7fee1806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.440+0000 7fee37fff700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee1806ac50 0x7fee1806d100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fee280011e0 tx=0x7fee28009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.444+0000 7fee34ff9700 1 -- 172.21.15.22:0/3525177201 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fee30054cb0 con 0x7fee3815a690 2023-07-06T01:19:17.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:17.643+0000 7fee3e75e700 1 -- 172.21.15.22:0/3525177201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fee380744b0 con 0x7fee3815a690 2023-07-06T01:19:18.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.573+0000 7fee34ff9700 1 -- 172.21.15.22:0/3525177201 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) v1 ==== 109+0+0 (secure 0 0 0) 0x7fee300582d0 con 0x7fee3815a690 2023-07-06T01:19:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.576+0000 7fee3e75e700 1 -- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee1806ac50 msgr2=0x7fee1806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:18.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.576+0000 7fee3e75e700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee1806ac50 0x7fee1806d100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fee280011e0 tx=0x7fee28009040 comp rx=0 tx=0).stop 2023-07-06T01:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.576+0000 7fee3e75e700 1 -- 172.21.15.22:0/3525177201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee3815a690 msgr2=0x7fee38155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.576+0000 7fee3e75e700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee3815a690 0x7fee38155790 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee3000efc0 tx=0x7fee3000c3a0 comp rx=0 tx=0).stop 2023-07-06T01:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.577+0000 7fee3e75e700 1 -- 172.21.15.22:0/3525177201 shutdown_connections 2023-07-06T01:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.577+0000 7fee3e75e700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee3815a690 0x7fee38155790 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.577+0000 7fee3e75e700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee380c9a40 0x7fee3814f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.577+0000 7fee3e75e700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee1806ac50 0x7fee1806d100 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:18.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.577+0000 7fee3e75e700 1 --2- 172.21.15.22:0/3525177201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee380cee60 0x7fee3815bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.577+0000 7fee3e75e700 1 -- 172.21.15.22:0/3525177201 >> 172.21.15.22:0/3525177201 conn(0x7fee380ccb00 msgr2=0x7fee38157410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.577+0000 7fee3e75e700 1 -- 172.21.15.22:0/3525177201 shutdown_connections 2023-07-06T01:19:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.577+0000 7fee3e75e700 1 -- 172.21.15.22:0/3525177201 wait complete. 2023-07-06T01:19:18.580 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'rbd' created 2023-07-06T01:19:18.594 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-07-06T01:19:18.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.725+0000 7f2beb09a700 1 Processor -- start 2023-07-06T01:19:18.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.725+0000 7f2beb09a700 1 -- start start 2023-07-06T01:19:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.726+0000 7f2beb09a700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f2be415a710 0x7f2be40ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.726+0000 7f2beb09a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2be40cef70 0x7f2be40cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.726+0000 7f2beb09a700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be40af270 con 0x7f2be40aecb0 2023-07-06T01:19:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.726+0000 7f2beb09a700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be40af550 con 0x7f2be40cef70 2023-07-06T01:19:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.726+0000 7f2beb09a700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be40af830 con 0x7f2be415a710 2023-07-06T01:19:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.726+0000 7f2be9637700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2be40cef70 0x7f2be40cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.726+0000 7f2be9637700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2be40cef70 0x7f2be40cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45026/0 (socket says 172.21.15.22:45026) 2023-07-06T01:19:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.726+0000 7f2be9637700 1 -- 172.21.15.22:0/3278658997 learned_addr learned my addr 172.21.15.22:0/3278658997 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.727+0000 7f2be8e36700 1 --2- 172.21.15.22:0/3278658997 >> v2:172.21.15.103:3300/0 conn(0x7f2be415a710 0x7f2be40ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.727+0000 7f2be9637700 1 -- 172.21.15.22:0/3278658997 >> v2:172.21.15.103:3300/0 conn(0x7f2be415a710 msgr2=0x7f2be40ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.727+0000 7f2be9637700 1 --2- 172.21.15.22:0/3278658997 >> v2:172.21.15.103:3300/0 conn(0x7f2be415a710 0x7f2be40ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.727+0000 7f2be9637700 1 -- 172.21.15.22:0/3278658997 >> v1:172.21.15.103:6789/0 conn(0x7f2be40aecb0 legacy=0x7f2be40ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.727+0000 7f2be9637700 1 -- 172.21.15.22:0/3278658997 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be40cb2e0 con 0x7f2be40cef70 2023-07-06T01:19:18.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.727+0000 7f2be9637700 1 --2- 172.21.15.22:0/3278658997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be40cef70 0x7f2be40cbae0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2be4070f00 tx=0x7f2bdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3660c7093dd42053 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:18.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.727+0000 7f2be37fe700 1 -- 172.21.15.22:0/3278658997 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc00e070 con 0x7f2be40cef70 2023-07-06T01:19:18.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.727+0000 7f2be37fe700 1 -- 172.21.15.22:0/3278658997 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc00b910 con 0x7f2be40cef70 2023-07-06T01:19:18.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.727+0000 7f2be37fe700 1 -- 172.21.15.22:0/3278658997 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc00bcc0 con 0x7f2be40cef70 2023-07-06T01:19:18.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.728+0000 7f2beb09a700 1 -- 172.21.15.22:0/3278658997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be40cef70 msgr2=0x7f2be40cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:18.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.728+0000 7f2beb09a700 1 --2- 172.21.15.22:0/3278658997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be40cef70 0x7f2be40cbae0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2be4070f00 tx=0x7f2bdc00b330 comp rx=0 tx=0).stop 2023-07-06T01:19:18.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.728+0000 7f2beb09a700 1 -- 172.21.15.22:0/3278658997 shutdown_connections 2023-07-06T01:19:18.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.728+0000 7f2beb09a700 1 --2- 172.21.15.22:0/3278658997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be40cef70 0x7f2be40cbae0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:18.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.728+0000 7f2beb09a700 1 --2- 172.21.15.22:0/3278658997 >> v2:172.21.15.103:3300/0 conn(0x7f2be415a710 0x7f2be40ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:18.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.728+0000 7f2beb09a700 1 -- 172.21.15.22:0/3278658997 >> 172.21.15.22:0/3278658997 conn(0x7f2be40999f0 msgr2=0x7f2be4099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:18.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.728+0000 7f2beb09a700 1 -- 172.21.15.22:0/3278658997 shutdown_connections 2023-07-06T01:19:18.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.728+0000 7f2beb09a700 1 -- 172.21.15.22:0/3278658997 wait complete. 2023-07-06T01:19:18.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.729+0000 7f2beb09a700 1 Processor -- start 2023-07-06T01:19:18.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.729+0000 7f2beb09a700 1 -- start start 2023-07-06T01:19:18.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2beb09a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be40aecb0 0x7f2be41ef680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:18.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2beb09a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2be40cef70 0x7f2be40b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:18.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2beb09a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be415a710 0x7f2be40b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:18.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2beb09a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40b2c00 con 0x7f2be415a710 2023-07-06T01:19:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2beb09a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40c9ad0 con 0x7f2be40aecb0 2023-07-06T01:19:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2beb09a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40c9db0 con 0x7f2be40cef70 2023-07-06T01:19:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2be9637700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be415a710 0x7f2be40b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2be9637700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be415a710 0x7f2be40b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45032/0 (socket says 172.21.15.22:45032) 2023-07-06T01:19:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2be8e36700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be40aecb0 0x7f2be41ef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2be9637700 1 -- 172.21.15.22:0/1836177473 learned_addr learned my addr 172.21.15.22:0/1836177473 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2be3fff700 1 --2- 172.21.15.22:0/1836177473 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2be40cef70 0x7f2be40b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2be9637700 1 -- 172.21.15.22:0/1836177473 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2be40cef70 msgr2=0x7f2be40b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:18.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2be9637700 1 --2- 172.21.15.22:0/1836177473 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2be40cef70 0x7f2be40b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:18.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2be9637700 1 -- 172.21.15.22:0/1836177473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be40aecb0 msgr2=0x7f2be41ef680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:18.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2be9637700 1 --2- 172.21.15.22:0/1836177473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be40aecb0 0x7f2be41ef680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:18.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2be9637700 1 -- 172.21.15.22:0/1836177473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc009b30 con 0x7f2be415a710 2023-07-06T01:19:18.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2be9637700 1 --2- 172.21.15.22:0/1836177473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be415a710 0x7f2be40b26c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00b370 tx=0x7f2bdc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:18.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.730+0000 7f2be17fa700 1 -- 172.21.15.22:0/1836177473 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc004880 con 0x7f2be415a710 2023-07-06T01:19:18.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.731+0000 7f2be17fa700 1 -- 172.21.15.22:0/1836177473 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc0099b0 con 0x7f2be415a710 2023-07-06T01:19:18.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.731+0000 7f2beb09a700 1 -- 172.21.15.22:0/1836177473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be40ca090 con 0x7f2be415a710 2023-07-06T01:19:18.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.731+0000 7f2be17fa700 1 -- 172.21.15.22:0/1836177473 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc01c680 con 0x7f2be415a710 2023-07-06T01:19:18.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.731+0000 7f2beb09a700 1 -- 172.21.15.22:0/1836177473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be40c97e0 con 0x7f2be415a710 2023-07-06T01:19:18.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.732+0000 7f2beb09a700 1 -- 172.21.15.22:0/1836177473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be40ca550 con 0x7f2be415a710 2023-07-06T01:19:18.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.732+0000 7f2be17fa700 1 -- 172.21.15.22:0/1836177473 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f2bdc01c7e0 con 0x7f2be415a710 2023-07-06T01:19:18.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.732+0000 7f2be17fa700 1 --2- 172.21.15.22:0/1836177473 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2bc806ac50 0x7f2bc806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:18.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.733+0000 7f2be17fa700 1 -- 172.21.15.22:0/1836177473 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5188+0+0 (secure 0 0 0) 0x7f2bdc08a890 con 0x7f2be415a710 2023-07-06T01:19:18.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.733+0000 7f2be8e36700 1 --2- 172.21.15.22:0/1836177473 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2bc806ac50 0x7f2bc806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:18.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.733+0000 7f2be8e36700 1 --2- 172.21.15.22:0/1836177473 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2bc806ac50 0x7f2bc806d100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009b40 tx=0x7f2bd8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:18.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.737+0000 7f2be17fa700 1 -- 172.21.15.22:0/1836177473 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2bdc055e10 con 0x7f2be415a710 2023-07-06T01:19:18.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:18.939+0000 7f2beb09a700 1 -- 172.21.15.22:0/1836177473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2be40744b0 con 0x7f2be415a710 2023-07-06T01:19:19.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.575+0000 7f2be17fa700 1 -- 172.21.15.22:0/1836177473 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v16) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2bdc059430 con 0x7f2be415a710 2023-07-06T01:19:19.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.578+0000 7f2beb09a700 1 -- 172.21.15.22:0/1836177473 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2bc806ac50 msgr2=0x7f2bc806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:19.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.578+0000 7f2beb09a700 1 --2- 172.21.15.22:0/1836177473 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2bc806ac50 0x7f2bc806d100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009b40 tx=0x7f2bd8009210 comp rx=0 tx=0).stop 2023-07-06T01:19:19.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.578+0000 7f2beb09a700 1 -- 172.21.15.22:0/1836177473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be415a710 msgr2=0x7f2be40b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:19.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.578+0000 7f2beb09a700 1 --2- 172.21.15.22:0/1836177473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be415a710 0x7f2be40b26c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00b370 tx=0x7f2bdc004ad0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.578+0000 7f2beb09a700 1 -- 172.21.15.22:0/1836177473 shutdown_connections 2023-07-06T01:19:19.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.578+0000 7f2beb09a700 1 --2- 172.21.15.22:0/1836177473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2be415a710 0x7f2be40b26c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.578+0000 7f2beb09a700 1 --2- 172.21.15.22:0/1836177473 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2bc806ac50 0x7f2bc806d100 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.578+0000 7f2beb09a700 1 --2- 172.21.15.22:0/1836177473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2be40aecb0 0x7f2be41ef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.578+0000 7f2beb09a700 1 --2- 172.21.15.22:0/1836177473 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2be40cef70 0x7f2be40b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.578+0000 7f2beb09a700 1 -- 172.21.15.22:0/1836177473 >> 172.21.15.22:0/1836177473 conn(0x7f2be40999f0 msgr2=0x7f2be40cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:19.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.578+0000 7f2beb09a700 1 -- 172.21.15.22:0/1836177473 shutdown_connections 2023-07-06T01:19:19.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.578+0000 7f2beb09a700 1 -- 172.21.15.22:0/1836177473 wait complete. 2023-07-06T01:19:19.582 INFO:teuthology.orchestra.run.smithi022.stderr:enabled application 'rbd' on pool 'rbd' 2023-07-06T01:19:19.598 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-07-06T01:19:19.599 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T01:19:19.599 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T01:19:19.601 INFO:tasks.ceph.mds.a:Started 2023-07-06T01:19:19.601 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T01:19:19.602 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T01:19:19.604 INFO:tasks.ceph.mds.c:Started 2023-07-06T01:19:19.604 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T01:19:19.604 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T01:19:19.607 INFO:tasks.ceph.mds.b:Started 2023-07-06T01:19:19.607 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T01:19:19.607 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T01:19:19.609 INFO:tasks.ceph.mds.d:Started 2023-07-06T01:19:19.609 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-07-06T01:19:19.610 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:19.700 INFO:tasks.ceph.mds.d.smithi103.stdout:starting mds.d at 2023-07-06T01:19:19.701 INFO:tasks.ceph.mds.b.smithi103.stdout:starting mds.b at 2023-07-06T01:19:19.714 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2023-07-06T01:19:19.718 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2023-07-06T01:19:19.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.754+0000 7ff4b0a26700 1 Processor -- start 2023-07-06T01:19:19.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.754+0000 7ff4b0a26700 1 -- start start 2023-07-06T01:19:19.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4b0a26700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff4ac0b36f0 0x7ff4ac0cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:19.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4b0a26700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff4ac0cf5a0 0x7ff4ac0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:19.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4b0a26700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4ac0b3b70 0x7ff4ac0d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:19.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4b0a26700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ac15cff0 con 0x7ff4ac0cf5a0 2023-07-06T01:19:19.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4b0a26700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ac0cebd0 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4b0a26700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac0cdfc0 con 0x7ff4ac0b3b70 2023-07-06T01:19:19.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4a9d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff4ac0cf5a0 0x7ff4ac0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:19.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4aa59c700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff4ac0b36f0 0x7ff4ac0cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:19.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4a9d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff4ac0cf5a0 0x7ff4ac0d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45068/0 (socket says 172.21.15.22:45068) 2023-07-06T01:19:19.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4aa59c700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff4ac0b36f0 0x7ff4ac0cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:33790/0 (socket says 172.21.15.22:33790) 2023-07-06T01:19:19.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4aad9d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4ac0b3b70 0x7ff4ac0d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:19.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4a9d9b700 1 -- 172.21.15.22:0/435681956 learned_addr learned my addr 172.21.15.22:0/435681956 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:19.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4a9d9b700 1 -- 172.21.15.22:0/435681956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4ac0b3b70 msgr2=0x7ff4ac0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:19.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4a9d9b700 1 --2- 172.21.15.22:0/435681956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4ac0b3b70 0x7ff4ac0d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4a9d9b700 1 -- 172.21.15.22:0/435681956 >> v2:172.21.15.103:3300/0 conn(0x7ff4ac0b36f0 msgr2=0x7ff4ac0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:19.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4a9d9b700 1 --2- 172.21.15.22:0/435681956 >> v2:172.21.15.103:3300/0 conn(0x7ff4ac0b36f0 0x7ff4ac0cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4a9d9b700 1 -- 172.21.15.22:0/435681956 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac0ce2a0 con 0x7ff4ac0cf5a0 2023-07-06T01:19:19.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4aa59c700 1 --2- 172.21.15.22:0/435681956 >> v2:172.21.15.103:3300/0 conn(0x7ff4ac0b36f0 0x7ff4ac0cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:19.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4aad9d700 1 --2- 172.21.15.22:0/435681956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4ac0b3b70 0x7ff4ac0d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:19.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.755+0000 7ff4a9d9b700 1 --2- 172.21.15.22:0/435681956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ac0cf5a0 0x7ff4ac0d0a90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac08b8d0 tx=0x7ff4a0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a884ea5c3a5b8ca4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:19.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.756+0000 7ff4a959a700 1 -- 172.21.15.22:0/435681956 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0011830 con 0x7ff4ac0cf5a0 2023-07-06T01:19:19.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.756+0000 7ff4a959a700 1 -- 172.21.15.22:0/435681956 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a0011990 con 0x7ff4ac0cf5a0 2023-07-06T01:19:19.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.756+0000 7ff4a959a700 1 -- 172.21.15.22:0/435681956 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0011d20 con 0x7ff4ac0cf5a0 2023-07-06T01:19:19.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.756+0000 7ff4b0a26700 1 -- 172.21.15.22:0/435681956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ac0cf5a0 msgr2=0x7ff4ac0d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:19.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.756+0000 7ff4b0a26700 1 --2- 172.21.15.22:0/435681956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ac0cf5a0 0x7ff4ac0d0a90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac08b8d0 tx=0x7ff4a0009bd0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.756+0000 7ff4b0a26700 1 -- 172.21.15.22:0/435681956 shutdown_connections 2023-07-06T01:19:19.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.756+0000 7ff4b0a26700 1 --2- 172.21.15.22:0/435681956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4ac0b3b70 0x7ff4ac0d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.756+0000 7ff4b0a26700 1 --2- 172.21.15.22:0/435681956 >> v2:172.21.15.103:3300/0 conn(0x7ff4ac0b36f0 0x7ff4ac0cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.756+0000 7ff4b0a26700 1 --2- 172.21.15.22:0/435681956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ac0cf5a0 0x7ff4ac0d0a90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.756+0000 7ff4b0a26700 1 -- 172.21.15.22:0/435681956 >> 172.21.15.22:0/435681956 conn(0x7ff4ac09c2e0 msgr2=0x7ff4ac0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.756+0000 7ff4b0a26700 1 -- 172.21.15.22:0/435681956 shutdown_connections 2023-07-06T01:19:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.756+0000 7ff4b0a26700 1 -- 172.21.15.22:0/435681956 wait complete. 2023-07-06T01:19:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.758+0000 7ff4b0a26700 1 Processor -- start 2023-07-06T01:19:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.758+0000 7ff4b0a26700 1 -- start start 2023-07-06T01:19:19.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4b0a26700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ac0b36f0 0x7ff4ac15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4b0a26700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4ac0b3b70 0x7ff4ac0b81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4b0a26700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4ac0cf5a0 0x7ff4ac0b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4b0a26700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac15e370 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4b0a26700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac15d2d0 con 0x7ff4ac0cf5a0 2023-07-06T01:19:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4b0a26700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac15d5b0 con 0x7ff4ac0b3b70 2023-07-06T01:19:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4aa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ac0b36f0 0x7ff4ac15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4aa59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ac0b36f0 0x7ff4ac15a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45072/0 (socket says 172.21.15.22:45072) 2023-07-06T01:19:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4aa59c700 1 -- 172.21.15.22:0/4160737523 learned_addr learned my addr 172.21.15.22:0/4160737523 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4a9d9b700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4ac0b3b70 0x7ff4ac0b81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4aad9d700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4ac0cf5a0 0x7ff4ac0b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4aa59c700 1 -- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4ac0b3b70 msgr2=0x7ff4ac0b81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4aa59c700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4ac0b3b70 0x7ff4ac0b81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4aa59c700 1 -- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4ac0cf5a0 msgr2=0x7ff4ac0b86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4aa59c700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4ac0cf5a0 0x7ff4ac0b86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4aa59c700 1 -- 172.21.15.22:0/4160737523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c0094a0 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4aa59c700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ac0b36f0 0x7ff4ac15a6d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00e980 tx=0x7ff49c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:19.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff49affd700 1 -- 172.21.15.22:0/4160737523 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0096e0 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff49affd700 1 -- 172.21.15.22:0/4160737523 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c00c640 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.759+0000 7ff4b0a26700 1 -- 172.21.15.22:0/4160737523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a4009a70 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.760+0000 7ff49affd700 1 -- 172.21.15.22:0/4160737523 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00c9f0 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.760+0000 7ff4b0a26700 1 -- 172.21.15.22:0/4160737523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a0009550 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.761+0000 7ff4b0a26700 1 -- 172.21.15.22:0/4160737523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ac0b4840 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.761+0000 7ff49affd700 1 -- 172.21.15.22:0/4160737523 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7ff49c00cb50 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.761+0000 7ff49affd700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff48c06ac50 0x7ff48c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.762+0000 7ff49affd700 1 -- 172.21.15.22:0/4160737523 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff49c088f90 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.762+0000 7ff4a9d9b700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff48c06ac50 0x7ff48c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:19.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.762+0000 7ff4a9d9b700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff48c06ac50 0x7ff48c06d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0cf9a0 tx=0x7ff4a0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.766+0000 7ff49affd700 1 -- 172.21.15.22:0/4160737523 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff49c054470 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.971+0000 7ff4b0a26700 1 -- 172.21.15.22:0/4160737523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4ac0c7810 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.972+0000 7ff49affd700 1 -- 172.21.15.22:0/4160737523 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14593 (secure 0 0 0) 0x7ff49c057a90 con 0x7ff4ac0b36f0 2023-07-06T01:19:19.974 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:19.974 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":16,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:19.572044+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.974+0000 7ff4b0a26700 1 -- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff48c06ac50 msgr2=0x7ff48c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.974+0000 7ff4b0a26700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff48c06ac50 0x7ff48c06d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0cf9a0 tx=0x7ff4a0008030 comp rx=0 tx=0).stop 2023-07-06T01:19:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.974+0000 7ff4b0a26700 1 -- 172.21.15.22:0/4160737523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ac0b36f0 msgr2=0x7ff4ac15a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.974+0000 7ff4b0a26700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ac0b36f0 0x7ff4ac15a6d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00e980 tx=0x7ff49c00ec90 comp rx=0 tx=0).stop 2023-07-06T01:19:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.974+0000 7ff4b0a26700 1 -- 172.21.15.22:0/4160737523 shutdown_connections 2023-07-06T01:19:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.974+0000 7ff4b0a26700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff48c06ac50 0x7ff48c06d100 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.974+0000 7ff4b0a26700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4ac0cf5a0 0x7ff4ac0b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.974+0000 7ff4b0a26700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ac0b36f0 0x7ff4ac15a6d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.974+0000 7ff4b0a26700 1 --2- 172.21.15.22:0/4160737523 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4ac0b3b70 0x7ff4ac0b81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:19.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.974+0000 7ff4b0a26700 1 -- 172.21.15.22:0/4160737523 >> 172.21.15.22:0/4160737523 conn(0x7ff4ac09c2e0 msgr2=0x7ff4ac15c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:19.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.974+0000 7ff4b0a26700 1 -- 172.21.15.22:0/4160737523 shutdown_connections 2023-07-06T01:19:19.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:19.974+0000 7ff4b0a26700 1 -- 172.21.15.22:0/4160737523 wait complete. 2023-07-06T01:19:19.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:19:19.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:19:20.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.118+0000 7f6dd6e9d700 1 Processor -- start 2023-07-06T01:19:20.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.118+0000 7f6dd6e9d700 1 -- start start 2023-07-06T01:19:20.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dd6e9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6dd015a690 0x7f6dd00ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dd6e9d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6dd00cee60 0x7f6dd006a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dd6e9d700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd00cd640 con 0x7f6dd00c9a40 2023-07-06T01:19:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dd6e9d700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd00cd920 con 0x7f6dd015a690 2023-07-06T01:19:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dd6e9d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00cdc00 con 0x7f6dd00cee60 2023-07-06T01:19:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dcffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6dd015a690 0x7f6dd00ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dd4c39700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6dd00c9a40 0x7f6dd00c9e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35824/0 (socket says 172.21.15.22:35824) 2023-07-06T01:19:20.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dd4c39700 1 -- 172.21.15.22:0/1364987435 learned_addr learned my addr 172.21.15.22:0/1364987435 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:20.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dd543a700 1 --2- 172.21.15.22:0/1364987435 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6dd00cee60 0x7f6dd006a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dcffff700 1 -- 172.21.15.22:0/1364987435 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6dd00cee60 msgr2=0x7f6dd006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:20.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dcffff700 1 --2- 172.21.15.22:0/1364987435 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6dd00cee60 0x7f6dd006a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dcffff700 1 -- 172.21.15.22:0/1364987435 >> v1:172.21.15.22:6789/0 conn(0x7f6dd00c9a40 legacy=0x7f6dd00c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dcffff700 1 -- 172.21.15.22:0/1364987435 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd00cdee0 con 0x7f6dd015a690 2023-07-06T01:19:20.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.119+0000 7f6dcf7fe700 1 -- 172.21.15.22:0/1364987435 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902500366 0 0) 0x7f6dd00cd640 con 0x7f6dd00c9a40 2023-07-06T01:19:20.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.120+0000 7f6dcffff700 1 --2- 172.21.15.22:0/1364987435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd015a690 0x7f6dd00ce920 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00bae0 tx=0x7f6dbc00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343e097ed8cb7499 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.120+0000 7f6dcf7fe700 1 -- 172.21.15.22:0/1364987435 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00dc70 con 0x7f6dd015a690 2023-07-06T01:19:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.120+0000 7f6dcf7fe700 1 -- 172.21.15.22:0/1364987435 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc00ddd0 con 0x7f6dd015a690 2023-07-06T01:19:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.120+0000 7f6dcf7fe700 1 -- 172.21.15.22:0/1364987435 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc010690 con 0x7f6dd015a690 2023-07-06T01:19:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.120+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/1364987435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd015a690 msgr2=0x7f6dd00ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.120+0000 7f6dd6e9d700 1 --2- 172.21.15.22:0/1364987435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd015a690 0x7f6dd00ce920 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00bae0 tx=0x7f6dbc00bdf0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.120+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/1364987435 shutdown_connections 2023-07-06T01:19:20.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.120+0000 7f6dd6e9d700 1 --2- 172.21.15.22:0/1364987435 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6dd00cee60 0x7f6dd006a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.120+0000 7f6dd6e9d700 1 --2- 172.21.15.22:0/1364987435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd015a690 0x7f6dd00ce920 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.120+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/1364987435 >> 172.21.15.22:0/1364987435 conn(0x7f6dd00ccb00 msgr2=0x7f6dd00cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.120+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/1364987435 shutdown_connections 2023-07-06T01:19:20.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.120+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/1364987435 wait complete. 2023-07-06T01:19:20.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.121+0000 7f6dd6e9d700 1 Processor -- start 2023-07-06T01:19:20.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.122+0000 7f6dd6e9d700 1 -- start start 2023-07-06T01:19:20.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.122+0000 7f6dd6e9d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6dd00c9a40 0x7f6dd0155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.122+0000 7f6dd6e9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd00cee60 0x7f6dd00ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.122+0000 7f6dd6e9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd015a690 0x7f6dd00acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.122+0000 7f6dd6e9d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00c30d0 con 0x7f6dd00cee60 2023-07-06T01:19:20.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.122+0000 7f6dd6e9d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00c8f00 con 0x7f6dd015a690 2023-07-06T01:19:20.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.122+0000 7f6dd6e9d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00c91a0 con 0x7f6dd00c9a40 2023-07-06T01:19:20.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.122+0000 7f6dcffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd00cee60 0x7f6dd00ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dcffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd00cee60 0x7f6dd00ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45088/0 (socket says 172.21.15.22:45088) 2023-07-06T01:19:20.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dcffff700 1 -- 172.21.15.22:0/2274816013 learned_addr learned my addr 172.21.15.22:0/2274816013 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:20.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dd4c39700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6dd00c9a40 0x7f6dd0155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dd543a700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd015a690 0x7f6dd00acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dcffff700 1 -- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6dd00c9a40 msgr2=0x7f6dd0155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dcffff700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6dd00c9a40 0x7f6dd0155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dcffff700 1 -- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd015a690 msgr2=0x7f6dd00acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dcffff700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd015a690 0x7f6dd00acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dcffff700 1 -- 172.21.15.22:0/2274816013 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc00b740 con 0x7f6dd00cee60 2023-07-06T01:19:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dcffff700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd00cee60 0x7f6dd00ac870 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00b2c0 tx=0x7f6dbc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dcd7fa700 1 -- 172.21.15.22:0/2274816013 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc003b70 con 0x7f6dd00cee60 2023-07-06T01:19:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dcd7fa700 1 -- 172.21.15.22:0/2274816013 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc003cd0 con 0x7f6dd00cee60 2023-07-06T01:19:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dcd7fa700 1 -- 172.21.15.22:0/2274816013 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc010670 con 0x7f6dd00cee60 2023-07-06T01:19:20.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/2274816013 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd00c3320 con 0x7f6dd00cee60 2023-07-06T01:19:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.123+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/2274816013 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd0156030 con 0x7f6dd00cee60 2023-07-06T01:19:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.124+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/2274816013 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd00744b0 con 0x7f6dd00cee60 2023-07-06T01:19:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.125+0000 7f6dcd7fa700 1 -- 172.21.15.22:0/2274816013 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f6dbc02c430 con 0x7f6dd00cee60 2023-07-06T01:19:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.125+0000 7f6dcd7fa700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6db406ac50 0x7f6db406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.125+0000 7f6dcd7fa700 1 -- 172.21.15.22:0/2274816013 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6dbc08a960 con 0x7f6dd00cee60 2023-07-06T01:19:20.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.129+0000 7f6dcd7fa700 1 -- 172.21.15.22:0/2274816013 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6dbc055ed0 con 0x7f6dd00cee60 2023-07-06T01:19:20.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.142+0000 7f6dd4c39700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6db406ac50 0x7f6db406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.142+0000 7f6dd4c39700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6db406ac50 0x7f6db406d100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4007fd0 tx=0x7f6dc4007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:20.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.329+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/2274816013 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6dd00b5370 con 0x7f6dd00cee60 2023-07-06T01:19:20.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.329+0000 7f6dcd7fa700 1 -- 172.21.15.22:0/2274816013 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6dbc0594f0 con 0x7f6dd00cee60 2023-07-06T01:19:20.331 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:19:20.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.331+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6db406ac50 msgr2=0x7f6db406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.331+0000 7f6dd6e9d700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6db406ac50 0x7f6db406d100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4007fd0 tx=0x7f6dc4007de0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.331+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/2274816013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd00cee60 msgr2=0x7f6dd00ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.331+0000 7f6dd6e9d700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd00cee60 0x7f6dd00ac870 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00b2c0 tx=0x7f6dbc004750 comp rx=0 tx=0).stop 2023-07-06T01:19:20.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.331+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/2274816013 shutdown_connections 2023-07-06T01:19:20.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.331+0000 7f6dd6e9d700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6db406ac50 0x7f6db406d100 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.331+0000 7f6dd6e9d700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6dd015a690 0x7f6dd00acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.331+0000 7f6dd6e9d700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6dd00c9a40 0x7f6dd0155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.331+0000 7f6dd6e9d700 1 --2- 172.21.15.22:0/2274816013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6dd00cee60 0x7f6dd00ac870 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.331+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/2274816013 >> 172.21.15.22:0/2274816013 conn(0x7f6dd00ccb00 msgr2=0x7f6dd00c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:20.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.331+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/2274816013 shutdown_connections 2023-07-06T01:19:20.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.331+0000 7f6dd6e9d700 1 -- 172.21.15.22:0/2274816013 wait complete. 2023-07-06T01:19:20.345 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:19:20.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.471+0000 7fd8c8dfe700 1 Processor -- start 2023-07-06T01:19:20.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.471+0000 7fd8c8dfe700 1 -- start start 2023-07-06T01:19:20.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.472+0000 7fd8c8dfe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd8c40ce860 0x7fd8c40cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.472+0000 7fd8c8dfe700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c40af270 con 0x7fd8c40cbae0 2023-07-06T01:19:20.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.472+0000 7fd8c8dfe700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c40af550 con 0x7fd8c415a710 2023-07-06T01:19:20.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.472+0000 7fd8c8dfe700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40cf210 con 0x7fd8c40ce860 2023-07-06T01:19:20.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.472+0000 7fd8c259c700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fd8c415a710 0x7fd8c40ae770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:37902/0 (socket says 172.21.15.22:37902) 2023-07-06T01:19:20.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.472+0000 7fd8c259c700 1 -- 172.21.15.22:0/583371476 learned_addr learned my addr 172.21.15.22:0/583371476 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:20.477 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.472+0000 7fd8c1d9b700 1 --2- 172.21.15.22:0/583371476 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd8c40ce860 0x7fd8c40cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.472+0000 7fd8c159a700 1 -- 172.21.15.22:0/583371476 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890965716 0 0) 0x7fd8c40af550 con 0x7fd8c415a710 2023-07-06T01:19:20.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.472+0000 7fd8c159a700 1 -- 172.21.15.22:0/583371476 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8ac0035b0 con 0x7fd8c415a710 2023-07-06T01:19:20.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.472+0000 7fd8c1d9b700 1 -- 172.21.15.22:0/583371476 >> v1:172.21.15.103:6789/0 conn(0x7fd8c415a710 legacy=0x7fd8c40ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.472+0000 7fd8c1d9b700 1 -- 172.21.15.22:0/583371476 >> v1:172.21.15.22:6789/0 conn(0x7fd8c40cbae0 legacy=0x7fd8c40cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.472+0000 7fd8c1d9b700 1 -- 172.21.15.22:0/583371476 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c40af830 con 0x7fd8c40ce860 2023-07-06T01:19:20.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.473+0000 7fd8c1d9b700 1 --2- 172.21.15.22:0/583371476 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd8c40ce860 0x7fd8c40cecd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009870 tx=0x7fd8b8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a363c3fb4833b93 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:20.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.473+0000 7fd8c159a700 1 -- 172.21.15.22:0/583371476 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8011830 con 0x7fd8c40ce860 2023-07-06T01:19:20.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.473+0000 7fd8c159a700 1 -- 172.21.15.22:0/583371476 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b8011990 con 0x7fd8c40ce860 2023-07-06T01:19:20.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.473+0000 7fd8c159a700 1 -- 172.21.15.22:0/583371476 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8011d20 con 0x7fd8c40ce860 2023-07-06T01:19:20.480 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.474+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/583371476 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd8c40ce860 msgr2=0x7fd8c40cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.474+0000 7fd8c8dfe700 1 --2- 172.21.15.22:0/583371476 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd8c40ce860 0x7fd8c40cecd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009870 tx=0x7fd8b8009b80 comp rx=0 tx=0).stop 2023-07-06T01:19:20.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.474+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/583371476 shutdown_connections 2023-07-06T01:19:20.481 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.474+0000 7fd8c8dfe700 1 --2- 172.21.15.22:0/583371476 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd8c40ce860 0x7fd8c40cecd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.474+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/583371476 >> 172.21.15.22:0/583371476 conn(0x7fd8c40999f0 msgr2=0x7fd8c4099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:20.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.474+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/583371476 shutdown_connections 2023-07-06T01:19:20.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.474+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/583371476 wait complete. 2023-07-06T01:19:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.475+0000 7fd8c8dfe700 1 Processor -- start 2023-07-06T01:19:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.475+0000 7fd8c8dfe700 1 -- start start 2023-07-06T01:19:20.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c8dfe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8c40cbae0 0x7fd8c40b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c8dfe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c40ce860 0x7fd8c40b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c8dfe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd8c415a710 0x7fd8c4149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c8dfe700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40b0580 con 0x7fd8c40ce860 2023-07-06T01:19:20.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c8dfe700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40b07b0 con 0x7fd8c40cbae0 2023-07-06T01:19:20.488 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c8dfe700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40b0a90 con 0x7fd8c415a710 2023-07-06T01:19:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c1d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c40ce860 0x7fd8c40b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c259c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8c40cbae0 0x7fd8c40b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c1d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c40ce860 0x7fd8c40b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45102/0 (socket says 172.21.15.22:45102) 2023-07-06T01:19:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c259c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8c40cbae0 0x7fd8c40b5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:33824/0 (socket says 172.21.15.22:33824) 2023-07-06T01:19:20.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c1d9b700 1 -- 172.21.15.22:0/2125457763 learned_addr learned my addr 172.21.15.22:0/2125457763 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:20.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c2d9d700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd8c415a710 0x7fd8c4149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c259c700 1 -- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd8c415a710 msgr2=0x7fd8c4149010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c259c700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd8c415a710 0x7fd8c4149010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c259c700 1 -- 172.21.15.22:0/2125457763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c40ce860 msgr2=0x7fd8c40b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c259c700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c40ce860 0x7fd8c40b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c259c700 1 -- 172.21.15.22:0/2125457763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b80094a0 con 0x7fd8c40cbae0 2023-07-06T01:19:20.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.476+0000 7fd8c2d9d700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd8c415a710 0x7fd8c4149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:20.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.477+0000 7fd8c259c700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8c40cbae0 0x7fd8c40b5700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40018e0 tx=0x7fd8b4001bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:20.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.477+0000 7fd8b2ffd700 1 -- 172.21.15.22:0/2125457763 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4013070 con 0x7fd8c40cbae0 2023-07-06T01:19:20.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.477+0000 7fd8b2ffd700 1 -- 172.21.15.22:0/2125457763 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b400e070 con 0x7fd8c40cbae0 2023-07-06T01:19:20.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.477+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/2125457763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c40b0d70 con 0x7fd8c40cbae0 2023-07-06T01:19:20.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.477+0000 7fd8b2ffd700 1 -- 172.21.15.22:0/2125457763 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4005920 con 0x7fd8c40cbae0 2023-07-06T01:19:20.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.477+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/2125457763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c40b1330 con 0x7fd8c40cbae0 2023-07-06T01:19:20.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.478+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/2125457763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c40744b0 con 0x7fd8c40cbae0 2023-07-06T01:19:20.491 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.478+0000 7fd8b2ffd700 1 -- 172.21.15.22:0/2125457763 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fd8b4005a80 con 0x7fd8c40cbae0 2023-07-06T01:19:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.479+0000 7fd8b2ffd700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd8a406ac50 0x7fd8a406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.479+0000 7fd8b2ffd700 1 -- 172.21.15.22:0/2125457763 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd8b408c630 con 0x7fd8c40cbae0 2023-07-06T01:19:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.479+0000 7fd8c1d9b700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd8a406ac50 0x7fd8a406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.479+0000 7fd8c1d9b700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd8a406ac50 0x7fd8a406d100 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40cd9c0 tx=0x7fd8b8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:20.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.483+0000 7fd8b2ffd700 1 -- 172.21.15.22:0/2125457763 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd8b4057bd0 con 0x7fd8c40cbae0 2023-07-06T01:19:20.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.686+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/2125457763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8c40b6b80 con 0x7fd8c40cbae0 2023-07-06T01:19:20.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.687+0000 7fd8b2ffd700 1 -- 172.21.15.22:0/2125457763 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8b408c030 con 0x7fd8c40cbae0 2023-07-06T01:19:20.689 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:19:20.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.689+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd8a406ac50 msgr2=0x7fd8a406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.689+0000 7fd8c8dfe700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd8a406ac50 0x7fd8a406d100 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40cd9c0 tx=0x7fd8b8008030 comp rx=0 tx=0).stop 2023-07-06T01:19:20.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.689+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8c40cbae0 msgr2=0x7fd8c40b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.689+0000 7fd8c8dfe700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8c40cbae0 0x7fd8c40b5700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40018e0 tx=0x7fd8b4001bf0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.689+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/2125457763 shutdown_connections 2023-07-06T01:19:20.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.689+0000 7fd8c8dfe700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd8c415a710 0x7fd8c4149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.689+0000 7fd8c8dfe700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd8a406ac50 0x7fd8a406d100 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.689+0000 7fd8c8dfe700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8c40cbae0 0x7fd8c40b5700 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.689+0000 7fd8c8dfe700 1 --2- 172.21.15.22:0/2125457763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8c40ce860 0x7fd8c40b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.689+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/2125457763 >> 172.21.15.22:0/2125457763 conn(0x7fd8c40999f0 msgr2=0x7fd8c40cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:20.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.689+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/2125457763 shutdown_connections 2023-07-06T01:19:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.689+0000 7fd8c8dfe700 1 -- 172.21.15.22:0/2125457763 wait complete. 2023-07-06T01:19:20.704 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:19:20.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.829+0000 7f37f5f3d700 1 Processor -- start 2023-07-06T01:19:20.831 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.829+0000 7f37f5f3d700 1 -- start start 2023-07-06T01:19:20.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37f5f3d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37f00ccd70 0x7f37f015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37f5f3d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f37f006d040 0x7f37f006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37f5f3d700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f00cf230 con 0x7f37f0159360 2023-07-06T01:19:20.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37f5f3d700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f015df20 con 0x7f37f006d040 2023-07-06T01:19:20.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37f5f3d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f00ce620 con 0x7f37f00ccd70 2023-07-06T01:19:20.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37effff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f37f006d040 0x7f37f006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37ef7fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f37f0159360 0x7f37f00cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35856/0 (socket says 172.21.15.22:35856) 2023-07-06T01:19:20.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37ef7fe700 1 -- 172.21.15.22:0/4007287905 learned_addr learned my addr 172.21.15.22:0/4007287905 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:20.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37eeffd700 1 --2- 172.21.15.22:0/4007287905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37f00ccd70 0x7f37f015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37ee7fc700 1 -- 172.21.15.22:0/4007287905 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626003123 0 0) 0x7f37f00cf230 con 0x7f37f0159360 2023-07-06T01:19:20.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37ee7fc700 1 -- 172.21.15.22:0/4007287905 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37d80035b0 con 0x7f37f0159360 2023-07-06T01:19:20.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37effff700 1 -- 172.21.15.22:0/4007287905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37f00ccd70 msgr2=0x7f37f015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:20.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37effff700 1 --2- 172.21.15.22:0/4007287905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37f00ccd70 0x7f37f015d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37effff700 1 -- 172.21.15.22:0/4007287905 >> v1:172.21.15.22:6789/0 conn(0x7f37f0159360 legacy=0x7f37f00cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37effff700 1 -- 172.21.15.22:0/4007287905 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f00ce900 con 0x7f37f006d040 2023-07-06T01:19:20.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37eeffd700 1 --2- 172.21.15.22:0/4007287905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37f00ccd70 0x7f37f015d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:20.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37effff700 1 --2- 172.21.15.22:0/4007287905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f006d040 0x7f37f006d4a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009d90 tx=0x7f37e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62930f6bbf95471 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:20.835 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37ee7fc700 1 -- 172.21.15.22:0/4007287905 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e800e070 con 0x7f37f006d040 2023-07-06T01:19:20.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37ee7fc700 1 -- 172.21.15.22:0/4007287905 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e800bb20 con 0x7f37f006d040 2023-07-06T01:19:20.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.830+0000 7f37ee7fc700 1 -- 172.21.15.22:0/4007287905 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e800be90 con 0x7f37f006d040 2023-07-06T01:19:20.836 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.831+0000 7f37f5f3d700 1 -- 172.21.15.22:0/4007287905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f006d040 msgr2=0x7f37f006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.831+0000 7f37f5f3d700 1 --2- 172.21.15.22:0/4007287905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f006d040 0x7f37f006d4a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009d90 tx=0x7f37e800b330 comp rx=0 tx=0).stop 2023-07-06T01:19:20.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.831+0000 7f37f5f3d700 1 -- 172.21.15.22:0/4007287905 shutdown_connections 2023-07-06T01:19:20.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.831+0000 7f37f5f3d700 1 --2- 172.21.15.22:0/4007287905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f006d040 0x7f37f006d4a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.831+0000 7f37f5f3d700 1 --2- 172.21.15.22:0/4007287905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37f00ccd70 0x7f37f015d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.831+0000 7f37f5f3d700 1 -- 172.21.15.22:0/4007287905 >> 172.21.15.22:0/4007287905 conn(0x7f37f00b1890 msgr2=0x7f37f00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:20.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.831+0000 7f37f5f3d700 1 -- 172.21.15.22:0/4007287905 shutdown_connections 2023-07-06T01:19:20.837 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.831+0000 7f37f5f3d700 1 -- 172.21.15.22:0/4007287905 wait complete. 2023-07-06T01:19:20.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.832+0000 7f37f5f3d700 1 Processor -- start 2023-07-06T01:19:20.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.832+0000 7f37f5f3d700 1 -- start start 2023-07-06T01:19:20.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.833+0000 7f37f5f3d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37f006d040 0x7f37f01ef450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.833+0000 7f37f5f3d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37f00ccd70 0x7f37f01f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.833+0000 7f37f5f3d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0159360 0x7f37f00b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.833+0000 7f37f5f3d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01f6410 con 0x7f37f0159360 2023-07-06T01:19:20.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.833+0000 7f37f5f3d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f014c9b0 con 0x7f37f006d040 2023-07-06T01:19:20.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.833+0000 7f37f5f3d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01ef990 con 0x7f37f00ccd70 2023-07-06T01:19:20.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.833+0000 7f37effff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0159360 0x7f37f00b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.833+0000 7f37effff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0159360 0x7f37f00b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45122/0 (socket says 172.21.15.22:45122) 2023-07-06T01:19:20.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.833+0000 7f37effff700 1 -- 172.21.15.22:0/3260998677 learned_addr learned my addr 172.21.15.22:0/3260998677 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:20.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.833+0000 7f37eeffd700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37f00ccd70 0x7f37f01f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.833+0000 7f37ef7fe700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37f006d040 0x7f37f01ef450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.834+0000 7f37effff700 1 -- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37f00ccd70 msgr2=0x7f37f01f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.834+0000 7f37effff700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37f00ccd70 0x7f37f01f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.834+0000 7f37effff700 1 -- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37f006d040 msgr2=0x7f37f01ef450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:20.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.834+0000 7f37effff700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37f006d040 0x7f37f01ef450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:20.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.834+0000 7f37effff700 1 -- 172.21.15.22:0/3260998677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e8009a70 con 0x7f37f0159360 2023-07-06T01:19:20.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.834+0000 7f37ef7fe700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37f006d040 0x7f37f01ef450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:20.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.834+0000 7f37effff700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0159360 0x7f37f00b3530 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f37f0071d00 tx=0x7f37e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:20.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.834+0000 7f37d7fff700 1 -- 172.21.15.22:0/3260998677 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8004960 con 0x7f37f0159360 2023-07-06T01:19:20.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.834+0000 7f37d7fff700 1 -- 172.21.15.22:0/3260998677 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e8020070 con 0x7f37f0159360 2023-07-06T01:19:20.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.834+0000 7f37f5f3d700 1 -- 172.21.15.22:0/3260998677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f01f5e50 con 0x7f37f0159360 2023-07-06T01:19:20.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.834+0000 7f37d7fff700 1 -- 172.21.15.22:0/3260998677 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e80276c0 con 0x7f37f0159360 2023-07-06T01:19:20.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.834+0000 7f37f5f3d700 1 -- 172.21.15.22:0/3260998677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f00b2d30 con 0x7f37f0159360 2023-07-06T01:19:20.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.835+0000 7f37f5f3d700 1 -- 172.21.15.22:0/3260998677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f00b2b90 con 0x7f37f0159360 2023-07-06T01:19:20.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.835+0000 7f37d7fff700 1 -- 172.21.15.22:0/3260998677 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f37e8012430 con 0x7f37f0159360 2023-07-06T01:19:20.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.836+0000 7f37d7fff700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f37dc06ace0 0x7f37dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:20.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.836+0000 7f37d7fff700 1 -- 172.21.15.22:0/3260998677 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f37e8022070 con 0x7f37f0159360 2023-07-06T01:19:20.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.836+0000 7f37ef7fe700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f37dc06ace0 0x7f37dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:20.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.836+0000 7f37ef7fe700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f37dc06ace0 0x7f37dc06d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f37e4005910 tx=0x7f37e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:20.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:20.840+0000 7f37d7fff700 1 -- 172.21.15.22:0/3260998677 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f37f00b2b90 con 0x7f37f0159360 2023-07-06T01:19:21.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.082+0000 7f37f5f3d700 1 -- 172.21.15.22:0/3260998677 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37f00b2b90 con 0x7f37f0159360 2023-07-06T01:19:21.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.082+0000 7f37d7fff700 1 -- 172.21.15.22:0/3260998677 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f37f00b2b90 con 0x7f37f0159360 2023-07-06T01:19:21.085 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:21.085 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/2496681609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2496681609},{"type":"v1","addr":"172.21.15.22:6833","nonce":2496681609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2891594418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2891594418},{"type":"v1","addr":"172.21.15.22:6835","nonce":2891594418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/4097705589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4097705589},{"type":"v1","addr":"172.21.15.103:6836","nonce":4097705589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/4015300665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":4015300665},{"type":"v1","addr":"172.21.15.103:6837","nonce":4015300665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-06T01:19:21.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.084+0000 7f37f5f3d700 1 -- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f37dc06ace0 msgr2=0x7f37dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.084+0000 7f37f5f3d700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f37dc06ace0 0x7f37dc06d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f37e4005910 tx=0x7f37e4009040 comp rx=0 tx=0).stop 2023-07-06T01:19:21.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.084+0000 7f37f5f3d700 1 -- 172.21.15.22:0/3260998677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0159360 msgr2=0x7f37f00b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.084+0000 7f37f5f3d700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0159360 0x7f37f00b3530 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f37f0071d00 tx=0x7f37e8004ad0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.085+0000 7f37f5f3d700 1 -- 172.21.15.22:0/3260998677 shutdown_connections 2023-07-06T01:19:21.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.085+0000 7f37f5f3d700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f37f0159360 0x7f37f00b3530 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.085+0000 7f37f5f3d700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f37dc06ace0 0x7f37dc06d190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.085+0000 7f37f5f3d700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37f006d040 0x7f37f01ef450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.085+0000 7f37f5f3d700 1 --2- 172.21.15.22:0/3260998677 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37f00ccd70 0x7f37f01f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.085+0000 7f37f5f3d700 1 -- 172.21.15.22:0/3260998677 >> 172.21.15.22:0/3260998677 conn(0x7f37f00b1890 msgr2=0x7f37f006ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.085+0000 7f37f5f3d700 1 -- 172.21.15.22:0/3260998677 shutdown_connections 2023-07-06T01:19:21.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.085+0000 7f37f5f3d700 1 -- 172.21.15.22:0/3260998677 wait complete. 2023-07-06T01:19:21.090 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 3 2023-07-06T01:19:21.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:21.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.221+0000 7fef3b98c700 1 Processor -- start 2023-07-06T01:19:21.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.221+0000 7fef3b98c700 1 -- start start 2023-07-06T01:19:21.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.222+0000 7fef3b98c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fef3406b320 0x7fef3406c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.222+0000 7fef3b98c700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fef3406c730 0x7fef340c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.222+0000 7fef3b98c700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef34071f30 con 0x7fef3415a610 2023-07-06T01:19:21.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.222+0000 7fef3b98c700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef3415a9e0 con 0x7fef3406b320 2023-07-06T01:19:21.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.222+0000 7fef3b98c700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef3406cbb0 con 0x7fef3406c730 2023-07-06T01:19:21.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.222+0000 7fef38f27700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fef3406b320 0x7fef3406c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.222+0000 7fef39728700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fef3415a610 0x7fef3415eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35870/0 (socket says 172.21.15.22:35870) 2023-07-06T01:19:21.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.222+0000 7fef39728700 1 -- 172.21.15.22:0/75506899 learned_addr learned my addr 172.21.15.22:0/75506899 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:21.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.222+0000 7fef38f27700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fef3406b320 0x7fef3406c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45128/0 (socket says 172.21.15.22:45128) 2023-07-06T01:19:21.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.222+0000 7fef39f29700 1 --2- 172.21.15.22:0/75506899 >> v2:172.21.15.103:3300/0 conn(0x7fef3406c730 0x7fef340c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.223+0000 7fef38f27700 1 -- 172.21.15.22:0/75506899 >> v2:172.21.15.103:3300/0 conn(0x7fef3406c730 msgr2=0x7fef340c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:21.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.223+0000 7fef38f27700 1 --2- 172.21.15.22:0/75506899 >> v2:172.21.15.103:3300/0 conn(0x7fef3406c730 0x7fef340c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.223+0000 7fef38f27700 1 -- 172.21.15.22:0/75506899 >> v1:172.21.15.22:6789/0 conn(0x7fef3415a610 legacy=0x7fef3415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.223+0000 7fef2bfff700 1 -- 172.21.15.22:0/75506899 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731393765 0 0) 0x7fef34071f30 con 0x7fef3415a610 2023-07-06T01:19:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.223+0000 7fef38f27700 1 -- 172.21.15.22:0/75506899 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3406b770 con 0x7fef3406b320 2023-07-06T01:19:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.223+0000 7fef39f29700 1 --2- 172.21.15.22:0/75506899 >> v2:172.21.15.103:3300/0 conn(0x7fef3406c730 0x7fef340c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.223+0000 7fef38f27700 1 --2- 172.21.15.22:0/75506899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3406b320 0x7fef3406c1f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fef34066320 tx=0x7fef30009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bdd39e402a03cb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.223+0000 7fef2bfff700 1 -- 172.21.15.22:0/75506899 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef30011830 con 0x7fef3406b320 2023-07-06T01:19:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.223+0000 7fef2bfff700 1 -- 172.21.15.22:0/75506899 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef30011990 con 0x7fef3406b320 2023-07-06T01:19:21.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.223+0000 7fef2bfff700 1 -- 172.21.15.22:0/75506899 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef30011d20 con 0x7fef3406b320 2023-07-06T01:19:21.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.223+0000 7fef3b98c700 1 -- 172.21.15.22:0/75506899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3406b320 msgr2=0x7fef3406c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.223+0000 7fef3b98c700 1 --2- 172.21.15.22:0/75506899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3406b320 0x7fef3406c1f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fef34066320 tx=0x7fef30009b20 comp rx=0 tx=0).stop 2023-07-06T01:19:21.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.224+0000 7fef3b98c700 1 -- 172.21.15.22:0/75506899 shutdown_connections 2023-07-06T01:19:21.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.224+0000 7fef3b98c700 1 --2- 172.21.15.22:0/75506899 >> v2:172.21.15.103:3300/0 conn(0x7fef3406c730 0x7fef340c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.224+0000 7fef3b98c700 1 --2- 172.21.15.22:0/75506899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3406b320 0x7fef3406c1f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.224+0000 7fef3b98c700 1 -- 172.21.15.22:0/75506899 >> 172.21.15.22:0/75506899 conn(0x7fef340ccb00 msgr2=0x7fef340cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:21.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.224+0000 7fef3b98c700 1 -- 172.21.15.22:0/75506899 shutdown_connections 2023-07-06T01:19:21.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.224+0000 7fef3b98c700 1 -- 172.21.15.22:0/75506899 wait complete. 2023-07-06T01:19:21.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.225+0000 7fef3b98c700 1 Processor -- start 2023-07-06T01:19:21.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.225+0000 7fef3b98c700 1 -- start start 2023-07-06T01:19:21.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef3b98c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef3406b320 0x7fef340b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef3b98c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef3406c730 0x7fef34156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef3b98c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3415a610 0x7fef340c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef3b98c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef341f0db0 con 0x7fef3415a610 2023-07-06T01:19:21.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef3b98c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef340b3dc0 con 0x7fef3406c730 2023-07-06T01:19:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef3b98c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef340b40a0 con 0x7fef3406b320 2023-07-06T01:19:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef39f29700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3415a610 0x7fef340c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef39f29700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3415a610 0x7fef340c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45136/0 (socket says 172.21.15.22:45136) 2023-07-06T01:19:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef39f29700 1 -- 172.21.15.22:0/2128666358 learned_addr learned my addr 172.21.15.22:0/2128666358 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef39728700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef3406b320 0x7fef340b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef38f27700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef3406c730 0x7fef34156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef39f29700 1 -- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef3406b320 msgr2=0x7fef340b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef39f29700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef3406b320 0x7fef340b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef39f29700 1 -- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef3406c730 msgr2=0x7fef34156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef39f29700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef3406c730 0x7fef34156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef39f29700 1 -- 172.21.15.22:0/2128666358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef300094a0 con 0x7fef3415a610 2023-07-06T01:19:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef39f29700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3415a610 0x7fef340c4d50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fef2c007ef0 tx=0x7fef2c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef29ffb700 1 -- 172.21.15.22:0/2128666358 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c00dd80 con 0x7fef3415a610 2023-07-06T01:19:21.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef29ffb700 1 -- 172.21.15.22:0/2128666358 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c00f040 con 0x7fef3415a610 2023-07-06T01:19:21.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef3b98c700 1 -- 172.21.15.22:0/2128666358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef340b4380 con 0x7fef3415a610 2023-07-06T01:19:21.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.226+0000 7fef29ffb700 1 -- 172.21.15.22:0/2128666358 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c003c60 con 0x7fef3415a610 2023-07-06T01:19:21.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.227+0000 7fef3b98c700 1 -- 172.21.15.22:0/2128666358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef340b50c0 con 0x7fef3415a610 2023-07-06T01:19:21.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.228+0000 7fef3b98c700 1 -- 172.21.15.22:0/2128666358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef340744b0 con 0x7fef3415a610 2023-07-06T01:19:21.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.228+0000 7fef29ffb700 1 -- 172.21.15.22:0/2128666358 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef2c004030 con 0x7fef3415a610 2023-07-06T01:19:21.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.229+0000 7fef29ffb700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef1806ace0 0x7fef1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.229+0000 7fef29ffb700 1 -- 172.21.15.22:0/2128666358 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fef2c089150 con 0x7fef3415a610 2023-07-06T01:19:21.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.229+0000 7fef39728700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef1806ace0 0x7fef1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.229+0000 7fef39728700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef1806ace0 0x7fef1806d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef24000f60 tx=0x7fef24007e70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:21.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.233+0000 7fef29ffb700 1 -- 172.21.15.22:0/2128666358 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fef2c0545e0 con 0x7fef3415a610 2023-07-06T01:19:21.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.432+0000 7fef3b98c700 1 -- 172.21.15.22:0/2128666358 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef340cfbd0 con 0x7fef3415a610 2023-07-06T01:19:21.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.433+0000 7fef29ffb700 1 -- 172.21.15.22:0/2128666358 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+14567 (secure 0 0 0) 0x7fef2c057c00 con 0x7fef3415a610 2023-07-06T01:19:21.434 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:21.435 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:20.580252+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:21.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.435+0000 7fef3b98c700 1 -- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef1806ace0 msgr2=0x7fef1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.435+0000 7fef3b98c700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef1806ace0 0x7fef1806d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef24000f60 tx=0x7fef24007e70 comp rx=0 tx=0).stop 2023-07-06T01:19:21.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.435+0000 7fef3b98c700 1 -- 172.21.15.22:0/2128666358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3415a610 msgr2=0x7fef340c4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.435+0000 7fef3b98c700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3415a610 0x7fef340c4d50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fef2c007ef0 tx=0x7fef2c00d3b0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.435+0000 7fef3b98c700 1 -- 172.21.15.22:0/2128666358 shutdown_connections 2023-07-06T01:19:21.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.435+0000 7fef3b98c700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef3415a610 0x7fef340c4d50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.435+0000 7fef3b98c700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef3406b320 0x7fef340b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.435+0000 7fef3b98c700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef1806ace0 0x7fef1806d190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.435+0000 7fef3b98c700 1 --2- 172.21.15.22:0/2128666358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef3406c730 0x7fef34156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.435+0000 7fef3b98c700 1 -- 172.21.15.22:0/2128666358 >> 172.21.15.22:0/2128666358 conn(0x7fef340ccb00 msgr2=0x7fef3406a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:21.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.435+0000 7fef3b98c700 1 -- 172.21.15.22:0/2128666358 shutdown_connections 2023-07-06T01:19:21.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.435+0000 7fef3b98c700 1 -- 172.21.15.22:0/2128666358 wait complete. 2023-07-06T01:19:21.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:19:21.449 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:19:21.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.575+0000 7fa80901b700 1 Processor -- start 2023-07-06T01:19:21.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.575+0000 7fa80901b700 1 -- start start 2023-07-06T01:19:21.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa80901b700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa804159360 0x7fa80415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa80901b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8040ccae0 0x7fa804071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa80901b700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8040cf230 con 0x7fa8040cc660 2023-07-06T01:19:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa80901b700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80415df20 con 0x7fa804159360 2023-07-06T01:19:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa80901b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8040ce620 con 0x7fa8040ccae0 2023-07-06T01:19:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa80259c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa8040cc660 0x7fa804070a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35886/0 (socket says 172.21.15.22:35886) 2023-07-06T01:19:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa80259c700 1 -- 172.21.15.22:0/172190826 learned_addr learned my addr 172.21.15.22:0/172190826 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa802d9d700 1 --2- 172.21.15.22:0/172190826 >> v2:172.21.15.103:3300/0 conn(0x7fa804159360 0x7fa80415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa80359e700 1 --2- 172.21.15.22:0/172190826 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8040ccae0 0x7fa804071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa801d9b700 1 -- 172.21.15.22:0/172190826 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864885260 0 0) 0x7fa8040cf230 con 0x7fa8040cc660 2023-07-06T01:19:21.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa801d9b700 1 -- 172.21.15.22:0/172190826 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7ec0035b0 con 0x7fa8040cc660 2023-07-06T01:19:21.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa801d9b700 1 -- 172.21.15.22:0/172190826 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 234897114 0 0) 0x7fa7ec0035b0 con 0x7fa8040cc660 2023-07-06T01:19:21.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa80359e700 1 -- 172.21.15.22:0/172190826 >> v2:172.21.15.103:3300/0 conn(0x7fa804159360 msgr2=0x7fa80415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:21.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa80359e700 1 --2- 172.21.15.22:0/172190826 >> v2:172.21.15.103:3300/0 conn(0x7fa804159360 0x7fa80415d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa80359e700 1 -- 172.21.15.22:0/172190826 >> v1:172.21.15.22:6789/0 conn(0x7fa8040cc660 legacy=0x7fa804070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa80359e700 1 -- 172.21.15.22:0/172190826 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8040ce900 con 0x7fa8040ccae0 2023-07-06T01:19:21.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.577+0000 7fa80359e700 1 --2- 172.21.15.22:0/172190826 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8040ccae0 0x7fa804071130 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009d90 tx=0x7fa7fc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aad2c22aba4a9c32 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:21.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.578+0000 7fa801d9b700 1 -- 172.21.15.22:0/172190826 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00e070 con 0x7fa8040ccae0 2023-07-06T01:19:21.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.578+0000 7fa801d9b700 1 -- 172.21.15.22:0/172190826 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc003a20 con 0x7fa8040ccae0 2023-07-06T01:19:21.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.578+0000 7fa801d9b700 1 -- 172.21.15.22:0/172190826 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4001670 con 0x7fa8040ccae0 2023-07-06T01:19:21.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.578+0000 7fa80901b700 1 -- 172.21.15.22:0/172190826 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8040ccae0 msgr2=0x7fa804071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.578+0000 7fa80901b700 1 --2- 172.21.15.22:0/172190826 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8040ccae0 0x7fa804071130 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009d90 tx=0x7fa7fc00b330 comp rx=0 tx=0).stop 2023-07-06T01:19:21.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.578+0000 7fa80901b700 1 -- 172.21.15.22:0/172190826 shutdown_connections 2023-07-06T01:19:21.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.578+0000 7fa80901b700 1 --2- 172.21.15.22:0/172190826 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8040ccae0 0x7fa804071130 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.578+0000 7fa80901b700 1 --2- 172.21.15.22:0/172190826 >> v2:172.21.15.103:3300/0 conn(0x7fa804159360 0x7fa80415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.578+0000 7fa80901b700 1 -- 172.21.15.22:0/172190826 >> 172.21.15.22:0/172190826 conn(0x7fa8040b1890 msgr2=0x7fa8040b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:21.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.578+0000 7fa80901b700 1 -- 172.21.15.22:0/172190826 shutdown_connections 2023-07-06T01:19:21.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.578+0000 7fa80901b700 1 -- 172.21.15.22:0/172190826 wait complete. 2023-07-06T01:19:21.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.580+0000 7fa80901b700 1 Processor -- start 2023-07-06T01:19:21.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.580+0000 7fa80901b700 1 -- start start 2023-07-06T01:19:21.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.580+0000 7fa80901b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8040cc660 0x7fa804153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.580+0000 7fa80901b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8040ccae0 0x7fa8040ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.580+0000 7fa80901b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa804159360 0x7fa8040b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.580+0000 7fa80901b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa804143e30 con 0x7fa8040cc660 2023-07-06T01:19:21.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.580+0000 7fa80901b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa804143b50 con 0x7fa8040ccae0 2023-07-06T01:19:21.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.580+0000 7fa80901b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa804153020 con 0x7fa804159360 2023-07-06T01:19:21.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.580+0000 7fa802d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8040cc660 0x7fa804153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.580+0000 7fa802d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8040cc660 0x7fa804153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45146/0 (socket says 172.21.15.22:45146) 2023-07-06T01:19:21.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.580+0000 7fa802d9d700 1 -- 172.21.15.22:0/2153294066 learned_addr learned my addr 172.21.15.22:0/2153294066 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:21.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.581+0000 7fa80259c700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8040ccae0 0x7fa8040ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.581+0000 7fa80359e700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa804159360 0x7fa8040b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.581+0000 7fa802d9d700 1 -- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa804159360 msgr2=0x7fa8040b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.581+0000 7fa802d9d700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa804159360 0x7fa8040b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.581+0000 7fa802d9d700 1 -- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8040ccae0 msgr2=0x7fa8040ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.581+0000 7fa802d9d700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8040ccae0 0x7fa8040ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.581+0000 7fa802d9d700 1 -- 172.21.15.22:0/2153294066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc009a70 con 0x7fa8040cc660 2023-07-06T01:19:21.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.581+0000 7fa802d9d700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8040cc660 0x7fa804153b00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800e870 tx=0x7fa7f800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:21.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.581+0000 7fa7eb7fe700 1 -- 172.21.15.22:0/2153294066 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8009590 con 0x7fa8040cc660 2023-07-06T01:19:21.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.581+0000 7fa7eb7fe700 1 -- 172.21.15.22:0/2153294066 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f800ee70 con 0x7fa8040cc660 2023-07-06T01:19:21.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.581+0000 7fa80901b700 1 -- 172.21.15.22:0/2153294066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa804153300 con 0x7fa8040cc660 2023-07-06T01:19:21.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.581+0000 7fa7eb7fe700 1 -- 172.21.15.22:0/2153294066 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f800ca50 con 0x7fa8040cc660 2023-07-06T01:19:21.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.581+0000 7fa80901b700 1 -- 172.21.15.22:0/2153294066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa804154040 con 0x7fa8040cc660 2023-07-06T01:19:21.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.582+0000 7fa80901b700 1 -- 172.21.15.22:0/2153294066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa804153950 con 0x7fa8040cc660 2023-07-06T01:19:21.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.583+0000 7fa7eb7fe700 1 -- 172.21.15.22:0/2153294066 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7f800cbb0 con 0x7fa8040cc660 2023-07-06T01:19:21.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.583+0000 7fa7eb7fe700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa7f006ace0 0x7fa7f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.583+0000 7fa7eb7fe700 1 -- 172.21.15.22:0/2153294066 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa7f8089100 con 0x7fa8040cc660 2023-07-06T01:19:21.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.583+0000 7fa80259c700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa7f006ace0 0x7fa7f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.584+0000 7fa80259c700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa7f006ace0 0x7fa7f006d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4001eb0 tx=0x7fa7f4001ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:21.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.587+0000 7fa7eb7fe700 1 -- 172.21.15.22:0/2153294066 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa7f8057f90 con 0x7fa8040cc660 2023-07-06T01:19:21.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.787+0000 7fa80901b700 1 -- 172.21.15.22:0/2153294066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa8040744b0 con 0x7fa8040cc660 2023-07-06T01:19:21.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.787+0000 7fa7eb7fe700 1 -- 172.21.15.22:0/2153294066 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa7f800ce60 con 0x7fa8040cc660 2023-07-06T01:19:21.789 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:19:21.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.789+0000 7fa80901b700 1 -- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa7f006ace0 msgr2=0x7fa7f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.789+0000 7fa80901b700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa7f006ace0 0x7fa7f006d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4001eb0 tx=0x7fa7f4001ce0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.789+0000 7fa80901b700 1 -- 172.21.15.22:0/2153294066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8040cc660 msgr2=0x7fa804153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.789+0000 7fa80901b700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8040cc660 0x7fa804153b00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800e870 tx=0x7fa7f800ec30 comp rx=0 tx=0).stop 2023-07-06T01:19:21.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.789+0000 7fa80901b700 1 -- 172.21.15.22:0/2153294066 shutdown_connections 2023-07-06T01:19:21.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.789+0000 7fa80901b700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa804159360 0x7fa8040b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.789+0000 7fa80901b700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8040cc660 0x7fa804153b00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.789+0000 7fa80901b700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa7f006ace0 0x7fa7f006d190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.789+0000 7fa80901b700 1 --2- 172.21.15.22:0/2153294066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa8040ccae0 0x7fa8040ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.789+0000 7fa80901b700 1 -- 172.21.15.22:0/2153294066 >> 172.21.15.22:0/2153294066 conn(0x7fa8040b1890 msgr2=0x7fa8040c65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:21.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.789+0000 7fa80901b700 1 -- 172.21.15.22:0/2153294066 shutdown_connections 2023-07-06T01:19:21.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.789+0000 7fa80901b700 1 -- 172.21.15.22:0/2153294066 wait complete. 2023-07-06T01:19:21.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:19:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.929+0000 7fb01b177700 1 Processor -- start 2023-07-06T01:19:21.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.929+0000 7fb01b177700 1 -- start start 2023-07-06T01:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb01b177700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fb01415c750 0x7fb0140cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb01b177700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb01406b820 0x7fb0140cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb01b177700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0140d06f0 con 0x7fb0140cdf80 2023-07-06T01:19:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb01b177700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0140ceb90 con 0x7fb01415c750 2023-07-06T01:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb01b177700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0140cee70 con 0x7fb01406b820 2023-07-06T01:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb019714700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb01406b820 0x7fb0140cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb018f13700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fb01415c750 0x7fb0140cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb019714700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb01406b820 0x7fb0140cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56296/0 (socket says 172.21.15.22:56296) 2023-07-06T01:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb019714700 1 -- 172.21.15.22:0/3062014316 learned_addr learned my addr 172.21.15.22:0/3062014316 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb019714700 1 -- 172.21.15.22:0/3062014316 >> v2:172.21.15.103:3300/0 conn(0x7fb01415c750 msgr2=0x7fb0140cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb019714700 1 --2- 172.21.15.22:0/3062014316 >> v2:172.21.15.103:3300/0 conn(0x7fb01415c750 0x7fb0140cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb019714700 1 -- 172.21.15.22:0/3062014316 >> v1:172.21.15.103:6789/0 conn(0x7fb0140cdf80 legacy=0x7fb01406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb019714700 1 -- 172.21.15.22:0/3062014316 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0140cf150 con 0x7fb01406b820 2023-07-06T01:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.930+0000 7fb018f13700 1 --2- 172.21.15.22:0/3062014316 >> v2:172.21.15.103:3300/0 conn(0x7fb01415c750 0x7fb0140cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:21.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.931+0000 7fb019714700 1 --2- 172.21.15.22:0/3062014316 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb01406b820 0x7fb0140cb2e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009e50 tx=0x7fb00c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f673ed330fe778ad server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:21.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.931+0000 7fb0137fe700 1 -- 172.21.15.22:0/3062014316 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00e070 con 0x7fb01406b820 2023-07-06T01:19:21.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.931+0000 7fb0137fe700 1 -- 172.21.15.22:0/3062014316 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c00b910 con 0x7fb01406b820 2023-07-06T01:19:21.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.931+0000 7fb0137fe700 1 -- 172.21.15.22:0/3062014316 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00bcc0 con 0x7fb01406b820 2023-07-06T01:19:21.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.931+0000 7fb01b177700 1 -- 172.21.15.22:0/3062014316 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb01406b820 msgr2=0x7fb0140cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.931+0000 7fb01b177700 1 --2- 172.21.15.22:0/3062014316 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb01406b820 0x7fb0140cb2e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009e50 tx=0x7fb00c00b330 comp rx=0 tx=0).stop 2023-07-06T01:19:21.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.931+0000 7fb01b177700 1 -- 172.21.15.22:0/3062014316 shutdown_connections 2023-07-06T01:19:21.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.931+0000 7fb01b177700 1 --2- 172.21.15.22:0/3062014316 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb01406b820 0x7fb0140cb2e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.931+0000 7fb01b177700 1 --2- 172.21.15.22:0/3062014316 >> v2:172.21.15.103:3300/0 conn(0x7fb01415c750 0x7fb0140cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.931+0000 7fb01b177700 1 -- 172.21.15.22:0/3062014316 >> 172.21.15.22:0/3062014316 conn(0x7fb0140999f0 msgr2=0x7fb014099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:21.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.931+0000 7fb01b177700 1 -- 172.21.15.22:0/3062014316 shutdown_connections 2023-07-06T01:19:21.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.931+0000 7fb01b177700 1 -- 172.21.15.22:0/3062014316 wait complete. 2023-07-06T01:19:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.933+0000 7fb01b177700 1 Processor -- start 2023-07-06T01:19:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.933+0000 7fb01b177700 1 -- start start 2023-07-06T01:19:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.933+0000 7fb01b177700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb01406b820 0x7fb0140b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.933+0000 7fb01b177700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb0140cdf80 0x7fb014157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.933+0000 7fb01b177700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb01415c750 0x7fb014158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.933+0000 7fb01b177700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0140b08f0 con 0x7fb01415c750 2023-07-06T01:19:21.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.933+0000 7fb01b177700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0140b0bd0 con 0x7fb01406b820 2023-07-06T01:19:21.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.933+0000 7fb01b177700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0140b0eb0 con 0x7fb0140cdf80 2023-07-06T01:19:21.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb019714700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb01415c750 0x7fb014158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb019714700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb01415c750 0x7fb014158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45164/0 (socket says 172.21.15.22:45164) 2023-07-06T01:19:21.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb019714700 1 -- 172.21.15.22:0/1160002633 learned_addr learned my addr 172.21.15.22:0/1160002633 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:21.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb018f13700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb01406b820 0x7fb0140b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb013fff700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb0140cdf80 0x7fb014157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb019714700 1 -- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb0140cdf80 msgr2=0x7fb014157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb019714700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb0140cdf80 0x7fb014157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb019714700 1 -- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb01406b820 msgr2=0x7fb0140b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb019714700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb01406b820 0x7fb0140b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb019714700 1 -- 172.21.15.22:0/1160002633 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0080094a0 con 0x7fb01415c750 2023-07-06T01:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb019714700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb01415c750 0x7fb014158390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb00c012000 tx=0x7fb00c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb0117fa700 1 -- 172.21.15.22:0/1160002633 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0048c0 con 0x7fb01415c750 2023-07-06T01:19:21.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb0117fa700 1 -- 172.21.15.22:0/1160002633 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c0092e0 con 0x7fb01415c750 2023-07-06T01:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb01b177700 1 -- 172.21.15.22:0/1160002633 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00c009b30 con 0x7fb01415c750 2023-07-06T01:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb0117fa700 1 -- 172.21.15.22:0/1160002633 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c01c680 con 0x7fb01415c750 2023-07-06T01:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.934+0000 7fb01b177700 1 -- 172.21.15.22:0/1160002633 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0140b1470 con 0x7fb01415c750 2023-07-06T01:19:21.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.935+0000 7fb01b177700 1 -- 172.21.15.22:0/1160002633 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0140744b0 con 0x7fb01415c750 2023-07-06T01:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.936+0000 7fb0117fa700 1 -- 172.21.15.22:0/1160002633 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb00c01c7e0 con 0x7fb01415c750 2023-07-06T01:19:21.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.936+0000 7fb0117fa700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7faff806ace0 0x7faff806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.936+0000 7fb0117fa700 1 -- 172.21.15.22:0/1160002633 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb00c08a8f0 con 0x7fb01415c750 2023-07-06T01:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.936+0000 7fb018f13700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7faff806ace0 0x7faff806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.937+0000 7fb018f13700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7faff806ace0 0x7faff806d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0140b62f0 tx=0x7fb008009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:21.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:21.940+0000 7fb0117fa700 1 -- 172.21.15.22:0/1160002633 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb00c055d80 con 0x7fb01415c750 2023-07-06T01:19:22.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.138+0000 7fb01b177700 1 -- 172.21.15.22:0/1160002633 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0140b3080 con 0x7fb01415c750 2023-07-06T01:19:22.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.138+0000 7fb0117fa700 1 -- 172.21.15.22:0/1160002633 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb00c0593a0 con 0x7fb01415c750 2023-07-06T01:19:22.141 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:19:22.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.140+0000 7fb01b177700 1 -- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7faff806ace0 msgr2=0x7faff806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:22.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.140+0000 7fb01b177700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7faff806ace0 0x7faff806d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0140b62f0 tx=0x7fb008009210 comp rx=0 tx=0).stop 2023-07-06T01:19:22.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.140+0000 7fb01b177700 1 -- 172.21.15.22:0/1160002633 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb01415c750 msgr2=0x7fb014158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:22.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.140+0000 7fb01b177700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb01415c750 0x7fb014158390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb00c012000 tx=0x7fb00c004ad0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.141+0000 7fb01b177700 1 -- 172.21.15.22:0/1160002633 shutdown_connections 2023-07-06T01:19:22.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.141+0000 7fb01b177700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb01415c750 0x7fb014158390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.141+0000 7fb01b177700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7faff806ace0 0x7faff806d190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.141+0000 7fb01b177700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb01406b820 0x7fb0140b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.141+0000 7fb01b177700 1 --2- 172.21.15.22:0/1160002633 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb0140cdf80 0x7fb014157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.141+0000 7fb01b177700 1 -- 172.21.15.22:0/1160002633 >> 172.21.15.22:0/1160002633 conn(0x7fb0140999f0 msgr2=0x7fb0140ae1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:22.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.141+0000 7fb01b177700 1 -- 172.21.15.22:0/1160002633 shutdown_connections 2023-07-06T01:19:22.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.141+0000 7fb01b177700 1 -- 172.21.15.22:0/1160002633 wait complete. 2023-07-06T01:19:22.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T01:19:22.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.277+0000 7f3a9cbda700 1 Processor -- start 2023-07-06T01:19:22.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.277+0000 7f3a9cbda700 1 -- start start 2023-07-06T01:19:22.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a9cbda700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3a9815a710 0x7f3a980ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:22.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a9cbda700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3a980cef70 0x7f3a980cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:22.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a9cbda700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a980af270 con 0x7f3a980aecb0 2023-07-06T01:19:22.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a9cbda700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a980af550 con 0x7f3a9815a710 2023-07-06T01:19:22.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a9cbda700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a980af830 con 0x7f3a980cef70 2023-07-06T01:19:22.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a9659c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3a9815a710 0x7f3a980ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:22.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a95d9b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f3a980aecb0 0x7f3a980ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35912/0 (socket says 172.21.15.22:35912) 2023-07-06T01:19:22.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a95d9b700 1 -- 172.21.15.22:0/635994838 learned_addr learned my addr 172.21.15.22:0/635994838 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:22.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a96d9d700 1 --2- 172.21.15.22:0/635994838 >> v2:172.21.15.103:3300/0 conn(0x7f3a980cef70 0x7f3a980cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:22.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a9659c700 1 -- 172.21.15.22:0/635994838 >> v2:172.21.15.103:3300/0 conn(0x7f3a980cef70 msgr2=0x7f3a980cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:22.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a9659c700 1 --2- 172.21.15.22:0/635994838 >> v2:172.21.15.103:3300/0 conn(0x7f3a980cef70 0x7f3a980cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a9659c700 1 -- 172.21.15.22:0/635994838 >> v1:172.21.15.22:6789/0 conn(0x7f3a980aecb0 legacy=0x7f3a980ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:22.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a9559a700 1 -- 172.21.15.22:0/635994838 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322178303 0 0) 0x7f3a980af270 con 0x7f3a980aecb0 2023-07-06T01:19:22.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a9659c700 1 -- 172.21.15.22:0/635994838 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a980cb2e0 con 0x7f3a9815a710 2023-07-06T01:19:22.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a96d9d700 1 --2- 172.21.15.22:0/635994838 >> v2:172.21.15.103:3300/0 conn(0x7f3a980cef70 0x7f3a980cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:22.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a9659c700 1 --2- 172.21.15.22:0/635994838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9815a710 0x7f3a980ae770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a98071c50 tx=0x7f3a88009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfbad8be090336b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:22.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.278+0000 7f3a9559a700 1 -- 172.21.15.22:0/635994838 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88011830 con 0x7f3a9815a710 2023-07-06T01:19:22.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.279+0000 7f3a9559a700 1 -- 172.21.15.22:0/635994838 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a88011990 con 0x7f3a9815a710 2023-07-06T01:19:22.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.279+0000 7f3a9559a700 1 -- 172.21.15.22:0/635994838 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88011d20 con 0x7f3a9815a710 2023-07-06T01:19:22.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.279+0000 7f3a9cbda700 1 -- 172.21.15.22:0/635994838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9815a710 msgr2=0x7f3a980ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.279+0000 7f3a9cbda700 1 --2- 172.21.15.22:0/635994838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9815a710 0x7f3a980ae770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a98071c50 tx=0x7f3a88009b20 comp rx=0 tx=0).stop 2023-07-06T01:19:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.279+0000 7f3a9cbda700 1 -- 172.21.15.22:0/635994838 shutdown_connections 2023-07-06T01:19:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.279+0000 7f3a9cbda700 1 --2- 172.21.15.22:0/635994838 >> v2:172.21.15.103:3300/0 conn(0x7f3a980cef70 0x7f3a980cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.279+0000 7f3a9cbda700 1 --2- 172.21.15.22:0/635994838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9815a710 0x7f3a980ae770 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.279+0000 7f3a9cbda700 1 -- 172.21.15.22:0/635994838 >> 172.21.15.22:0/635994838 conn(0x7f3a980999f0 msgr2=0x7f3a98099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.279+0000 7f3a9cbda700 1 -- 172.21.15.22:0/635994838 shutdown_connections 2023-07-06T01:19:22.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.279+0000 7f3a9cbda700 1 -- 172.21.15.22:0/635994838 wait complete. 2023-07-06T01:19:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.280+0000 7f3a9cbda700 1 Processor -- start 2023-07-06T01:19:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.280+0000 7f3a9cbda700 1 -- start start 2023-07-06T01:19:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.281+0000 7f3a9cbda700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3a980aecb0 0x7f3a981562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:22.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.281+0000 7f3a9cbda700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a980cef70 0x7f3a9806c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.281+0000 7f3a9cbda700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9815a710 0x7f3a980c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.281+0000 7f3a9cbda700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a98156830 con 0x7f3a9815a710 2023-07-06T01:19:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.281+0000 7f3a9cbda700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a980b2d00 con 0x7f3a980cef70 2023-07-06T01:19:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.281+0000 7f3a9cbda700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a980b20f0 con 0x7f3a980aecb0 2023-07-06T01:19:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.281+0000 7f3a96d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9815a710 0x7f3a980c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:22.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.281+0000 7f3a96d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9815a710 0x7f3a980c5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45178/0 (socket says 172.21.15.22:45178) 2023-07-06T01:19:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.281+0000 7f3a96d9d700 1 -- 172.21.15.22:0/1683896970 learned_addr learned my addr 172.21.15.22:0/1683896970 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.281+0000 7f3a95d9b700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a980cef70 0x7f3a9806c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.281+0000 7f3a9659c700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3a980aecb0 0x7f3a981562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.282+0000 7f3a96d9d700 1 -- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3a980aecb0 msgr2=0x7f3a981562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.282+0000 7f3a96d9d700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3a980aecb0 0x7f3a981562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.282+0000 7f3a96d9d700 1 -- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a980cef70 msgr2=0x7f3a9806c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.282+0000 7f3a96d9d700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a980cef70 0x7f3a9806c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.282+0000 7f3a96d9d700 1 -- 172.21.15.22:0/1683896970 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a880094a0 con 0x7f3a9815a710 2023-07-06T01:19:22.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.282+0000 7f3a9659c700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3a980aecb0 0x7f3a981562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:22.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.282+0000 7f3a96d9d700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9815a710 0x7f3a980c5ae0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000ce80 tx=0x7f3a9000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:22.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.282+0000 7f3a86ffd700 1 -- 172.21.15.22:0/1683896970 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000ae20 con 0x7f3a9815a710 2023-07-06T01:19:22.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.282+0000 7f3a86ffd700 1 -- 172.21.15.22:0/1683896970 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a90015440 con 0x7f3a9815a710 2023-07-06T01:19:22.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.282+0000 7f3a9cbda700 1 -- 172.21.15.22:0/1683896970 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a980b23d0 con 0x7f3a9815a710 2023-07-06T01:19:22.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.282+0000 7f3a86ffd700 1 -- 172.21.15.22:0/1683896970 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a900157f0 con 0x7f3a9815a710 2023-07-06T01:19:22.290 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.282+0000 7f3a9cbda700 1 -- 172.21.15.22:0/1683896970 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a980b2990 con 0x7f3a9815a710 2023-07-06T01:19:22.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.283+0000 7f3a9cbda700 1 -- 172.21.15.22:0/1683896970 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a980744b0 con 0x7f3a9815a710 2023-07-06T01:19:22.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.283+0000 7f3a86ffd700 1 -- 172.21.15.22:0/1683896970 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a9000d430 con 0x7f3a9815a710 2023-07-06T01:19:22.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.284+0000 7f3a86ffd700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3a7806ace0 0x7f3a7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:22.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.284+0000 7f3a86ffd700 1 -- 172.21.15.22:0/1683896970 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3a90089740 con 0x7f3a9815a710 2023-07-06T01:19:22.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.284+0000 7f3a9659c700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3a7806ace0 0x7f3a7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:22.291 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.285+0000 7f3a9659c700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3a7806ace0 0x7f3a7806d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3a880090b0 tx=0x7f3a8801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:22.292 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.289+0000 7f3a86ffd700 1 -- 172.21.15.22:0/1683896970 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3a90054b40 con 0x7f3a9815a710 2023-07-06T01:19:22.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.487+0000 7f3a9cbda700 1 -- 172.21.15.22:0/1683896970 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3a980b2e40 con 0x7f3a9815a710 2023-07-06T01:19:22.489 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.487+0000 7f3a86ffd700 1 -- 172.21.15.22:0/1683896970 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7f3a9001a090 con 0x7f3a9815a710 2023-07-06T01:19:22.489 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:22.490 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2023-07-06T01:19:22.490 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:19:22.490 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2023-07-06T01:19:22.490 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": "rbd" 2023-07-06T01:19:22.490 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:19:22.490 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:19:22.490 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 2, 2023-07-06T01:19:22.491 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2023-07-06T01:19:22.491 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:19:22.491 INFO:teuthology.orchestra.run.smithi022.stdout:] 2023-07-06T01:19:22.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.489+0000 7f3a9cbda700 1 -- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3a7806ace0 msgr2=0x7f3a7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:22.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.489+0000 7f3a9cbda700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3a7806ace0 0x7f3a7806d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3a880090b0 tx=0x7f3a8801c040 comp rx=0 tx=0).stop 2023-07-06T01:19:22.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.489+0000 7f3a9cbda700 1 -- 172.21.15.22:0/1683896970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9815a710 msgr2=0x7f3a980c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:22.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.489+0000 7f3a9cbda700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9815a710 0x7f3a980c5ae0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000ce80 tx=0x7f3a9000cf60 comp rx=0 tx=0).stop 2023-07-06T01:19:22.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.490+0000 7f3a9cbda700 1 -- 172.21.15.22:0/1683896970 shutdown_connections 2023-07-06T01:19:22.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.490+0000 7f3a9cbda700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a9815a710 0x7f3a980c5ae0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.490+0000 7f3a9cbda700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3a980aecb0 0x7f3a981562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.490+0000 7f3a9cbda700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3a7806ace0 0x7f3a7806d190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.490+0000 7f3a9cbda700 1 --2- 172.21.15.22:0/1683896970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a980cef70 0x7f3a9806c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.490+0000 7f3a9cbda700 1 -- 172.21.15.22:0/1683896970 >> 172.21.15.22:0/1683896970 conn(0x7f3a980999f0 msgr2=0x7f3a980cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.490+0000 7f3a9cbda700 1 -- 172.21.15.22:0/1683896970 shutdown_connections 2023-07-06T01:19:22.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.490+0000 7f3a9cbda700 1 -- 172.21.15.22:0/1683896970 wait complete. 2023-07-06T01:19:22.503 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T01:19:22.504 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T01:19:22.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.631+0000 7fbf0d441700 1 Processor -- start 2023-07-06T01:19:22.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.631+0000 7fbf0d441700 1 -- start start 2023-07-06T01:19:22.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.632+0000 7fbf0d441700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fbf080af2a0 0x7fbf0806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:22.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.632+0000 7fbf0d441700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf0806a000 0x7fbf0806a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:22.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.632+0000 7fbf0d441700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf080af670 con 0x7fbf080cbc10 2023-07-06T01:19:22.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.632+0000 7fbf0d441700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf0806a9e0 con 0x7fbf080af2a0 2023-07-06T01:19:22.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.632+0000 7fbf0d441700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080cc7a0 con 0x7fbf0806a000 2023-07-06T01:19:22.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.632+0000 7fbf077fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fbf080cbc10 0x7fbf080cc090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35930/0 (socket says 172.21.15.22:35930) 2023-07-06T01:19:22.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.632+0000 7fbf077fe700 1 -- 172.21.15.22:0/2998777447 learned_addr learned my addr 172.21.15.22:0/2998777447 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:22.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.632+0000 7fbf06ffd700 1 --2- 172.21.15.22:0/2998777447 >> v2:172.21.15.103:3300/0 conn(0x7fbf080af2a0 0x7fbf0806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:22.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.632+0000 7fbf067fc700 1 --2- 172.21.15.22:0/2998777447 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf0806a000 0x7fbf0806a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:22.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.632+0000 7fbf05ffb700 1 -- 172.21.15.22:0/2998777447 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572009348 0 0) 0x7fbf080af670 con 0x7fbf080cbc10 2023-07-06T01:19:22.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.632+0000 7fbf05ffb700 1 -- 172.21.15.22:0/2998777447 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbef00035b0 con 0x7fbf080cbc10 2023-07-06T01:19:22.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.632+0000 7fbf05ffb700 1 -- 172.21.15.22:0/2998777447 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 589366725 0 0) 0x7fbef00035b0 con 0x7fbf080cbc10 2023-07-06T01:19:22.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf06ffd700 1 -- 172.21.15.22:0/2998777447 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf0806a000 msgr2=0x7fbf0806a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:22.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf06ffd700 1 --2- 172.21.15.22:0/2998777447 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf0806a000 0x7fbf0806a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf06ffd700 1 -- 172.21.15.22:0/2998777447 >> v1:172.21.15.22:6789/0 conn(0x7fbf080cbc10 legacy=0x7fbf080cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf06ffd700 1 -- 172.21.15.22:0/2998777447 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0806d030 con 0x7fbf080af2a0 2023-07-06T01:19:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf067fc700 1 --2- 172.21.15.22:0/2998777447 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf0806a000 0x7fbf0806a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf06ffd700 1 --2- 172.21.15.22:0/2998777447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf080af2a0 0x7fbf0806caf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009840 tx=0x7fbefc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc01f9bddb648e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf05ffb700 1 -- 172.21.15.22:0/2998777447 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf000019f0 con 0x7fbf080af2a0 2023-07-06T01:19:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf05ffb700 1 -- 172.21.15.22:0/2998777447 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbefc004030 con 0x7fbf080af2a0 2023-07-06T01:19:22.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf05ffb700 1 -- 172.21.15.22:0/2998777447 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0043c0 con 0x7fbf080af2a0 2023-07-06T01:19:22.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf0d441700 1 -- 172.21.15.22:0/2998777447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf080af2a0 msgr2=0x7fbf0806caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:22.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf0d441700 1 --2- 172.21.15.22:0/2998777447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf080af2a0 0x7fbf0806caf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009840 tx=0x7fbefc009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:22.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf0d441700 1 -- 172.21.15.22:0/2998777447 shutdown_connections 2023-07-06T01:19:22.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf0d441700 1 --2- 172.21.15.22:0/2998777447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf080af2a0 0x7fbf0806caf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf0d441700 1 --2- 172.21.15.22:0/2998777447 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf0806a000 0x7fbf0806a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.633+0000 7fbf0d441700 1 -- 172.21.15.22:0/2998777447 >> 172.21.15.22:0/2998777447 conn(0x7fbf080b1890 msgr2=0x7fbf080b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:22.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.634+0000 7fbf0d441700 1 -- 172.21.15.22:0/2998777447 shutdown_connections 2023-07-06T01:19:22.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.634+0000 7fbf0d441700 1 -- 172.21.15.22:0/2998777447 wait complete. 2023-07-06T01:19:22.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.635+0000 7fbf0d441700 1 Processor -- start 2023-07-06T01:19:22.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.635+0000 7fbf0d441700 1 -- start start 2023-07-06T01:19:22.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf0d441700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf0806a000 0x7fbf080c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:22.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf0d441700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf080af2a0 0x7fbf080b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf0d441700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf080cbc10 0x7fbf0815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf0d441700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0815ae30 con 0x7fbf080af2a0 2023-07-06T01:19:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf0d441700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080b5be0 con 0x7fbf0806a000 2023-07-06T01:19:22.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf0d441700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080b5410 con 0x7fbf080cbc10 2023-07-06T01:19:22.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf067fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf080af2a0 0x7fbf080b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:22.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf067fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf080af2a0 0x7fbf080b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45190/0 (socket says 172.21.15.22:45190) 2023-07-06T01:19:22.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf067fc700 1 -- 172.21.15.22:0/3645470205 learned_addr learned my addr 172.21.15.22:0/3645470205 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:22.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf06ffd700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf0806a000 0x7fbf080c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:22.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf077fe700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf080cbc10 0x7fbf0815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:22.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf067fc700 1 -- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf080cbc10 msgr2=0x7fbf0815f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:22.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf067fc700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf080cbc10 0x7fbf0815f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf067fc700 1 -- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf0806a000 msgr2=0x7fbf080c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:22.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf067fc700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf0806a000 0x7fbf080c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:22.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf067fc700 1 -- 172.21.15.22:0/3645470205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef80094a0 con 0x7fbf080af2a0 2023-07-06T01:19:22.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf067fc700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf080af2a0 0x7fbf080b2b90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbef800ebe0 tx=0x7fbef8009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:22.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbeef7fe700 1 -- 172.21.15.22:0/3645470205 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef800c910 con 0x7fbf080af2a0 2023-07-06T01:19:22.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbeef7fe700 1 -- 172.21.15.22:0/3645470205 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef800ca70 con 0x7fbf080af2a0 2023-07-06T01:19:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbeef7fe700 1 -- 172.21.15.22:0/3645470205 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef800cde0 con 0x7fbf080af2a0 2023-07-06T01:19:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf0d441700 1 -- 172.21.15.22:0/3645470205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbefc0094a0 con 0x7fbf080af2a0 2023-07-06T01:19:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.636+0000 7fbf0d441700 1 -- 172.21.15.22:0/3645470205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf080b5900 con 0x7fbf080af2a0 2023-07-06T01:19:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.637+0000 7fbf0d441700 1 -- 172.21.15.22:0/3645470205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf080744b0 con 0x7fbf080af2a0 2023-07-06T01:19:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.638+0000 7fbeef7fe700 1 -- 172.21.15.22:0/3645470205 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbef8013470 con 0x7fbf080af2a0 2023-07-06T01:19:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.638+0000 7fbeef7fe700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbef406ace0 0x7fbef406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:22.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.638+0000 7fbeef7fe700 1 -- 172.21.15.22:0/3645470205 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbef80892f0 con 0x7fbf080af2a0 2023-07-06T01:19:22.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.638+0000 7fbf06ffd700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbef406ace0 0x7fbef406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:22.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.639+0000 7fbf06ffd700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbef406ace0 0x7fbef406d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0090b0 tx=0x7fbefc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:22.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.642+0000 7fbeef7fe700 1 -- 172.21.15.22:0/3645470205 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbef8054780 con 0x7fbf080af2a0 2023-07-06T01:19:22.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:22.842+0000 7fbf0d441700 1 -- 172.21.15.22:0/3645470205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbf0815fcf0 con 0x7fbf080af2a0 2023-07-06T01:19:23.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.622+0000 7fbeef7fe700 1 -- 172.21.15.22:0/3645470205 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbef8057da0 con 0x7fbf080af2a0 2023-07-06T01:19:23.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.627+0000 7fbf0d441700 1 -- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbef406ace0 msgr2=0x7fbef406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:23.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.627+0000 7fbf0d441700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbef406ace0 0x7fbef406d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0090b0 tx=0x7fbefc008030 comp rx=0 tx=0).stop 2023-07-06T01:19:23.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.627+0000 7fbf0d441700 1 -- 172.21.15.22:0/3645470205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf080af2a0 msgr2=0x7fbf080b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:23.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.627+0000 7fbf0d441700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf080af2a0 0x7fbf080b2b90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbef800ebe0 tx=0x7fbef8009ec0 comp rx=0 tx=0).stop 2023-07-06T01:19:23.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.627+0000 7fbf0d441700 1 -- 172.21.15.22:0/3645470205 shutdown_connections 2023-07-06T01:19:23.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.627+0000 7fbf0d441700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf080cbc10 0x7fbf0815f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:23.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.627+0000 7fbf0d441700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbef406ace0 0x7fbef406d190 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0090b0 tx=0x7fbefc008030 comp rx=0 tx=0).stop 2023-07-06T01:19:23.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.627+0000 7fbf0d441700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf0806a000 0x7fbf080c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:23.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.627+0000 7fbf0d441700 1 --2- 172.21.15.22:0/3645470205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf080af2a0 0x7fbf080b2b90 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbef800ebe0 tx=0x7fbef8009ec0 comp rx=0 tx=0).stop 2023-07-06T01:19:23.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.627+0000 7fbf0d441700 1 -- 172.21.15.22:0/3645470205 >> 172.21.15.22:0/3645470205 conn(0x7fbf080b1890 msgr2=0x7fbf0815bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:23.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.628+0000 7fbf0d441700 1 -- 172.21.15.22:0/3645470205 shutdown_connections 2023-07-06T01:19:23.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.628+0000 7fbf0d441700 1 -- 172.21.15.22:0/3645470205 wait complete. 2023-07-06T01:19:23.631 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2023-07-06T01:19:23.646 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T01:19:23.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.782+0000 7fe80cd81700 1 Processor -- start 2023-07-06T01:19:23.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.782+0000 7fe80cd81700 1 -- start start 2023-07-06T01:19:23.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe80cd81700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe8080d1520 0x7fe80806c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe80cd81700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe80806b110 0x7fe8080cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe80cd81700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80806b660 con 0x7fe80815cd60 2023-07-06T01:19:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe80cd81700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe80806c2e0 con 0x7fe8080d1520 2023-07-06T01:19:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe80cd81700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80806d0c0 con 0x7fe80806b110 2023-07-06T01:19:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe805d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe8080d1520 0x7fe80806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe80659c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe80815cd60 0x7fe8080d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35940/0 (socket says 172.21.15.22:35940) 2023-07-06T01:19:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe80659c700 1 -- 172.21.15.22:0/2723277413 learned_addr learned my addr 172.21.15.22:0/2723277413 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe805d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe8080d1520 0x7fe80806c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45198/0 (socket says 172.21.15.22:45198) 2023-07-06T01:19:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe806d9d700 1 --2- 172.21.15.22:0/2723277413 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe80806b110 0x7fe8080cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe805d9b700 1 -- 172.21.15.22:0/2723277413 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe80806b110 msgr2=0x7fe8080cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe805d9b700 1 --2- 172.21.15.22:0/2723277413 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe80806b110 0x7fe8080cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe805d9b700 1 -- 172.21.15.22:0/2723277413 >> v1:172.21.15.22:6789/0 conn(0x7fe80815cd60 legacy=0x7fe8080d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe80559a700 1 -- 172.21.15.22:0/2723277413 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602103022 0 0) 0x7fe80806b660 con 0x7fe80815cd60 2023-07-06T01:19:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.783+0000 7fe805d9b700 1 -- 172.21.15.22:0/2723277413 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80806b940 con 0x7fe8080d1520 2023-07-06T01:19:23.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.784+0000 7fe805d9b700 1 --2- 172.21.15.22:0/2723277413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8080d1520 0x7fe80806c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009840 tx=0x7fe7f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a506c2c629bbfa9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:23.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.784+0000 7fe80559a700 1 -- 172.21.15.22:0/2723277413 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8011830 con 0x7fe8080d1520 2023-07-06T01:19:23.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.784+0000 7fe80559a700 1 -- 172.21.15.22:0/2723277413 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f8011990 con 0x7fe8080d1520 2023-07-06T01:19:23.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.784+0000 7fe80559a700 1 -- 172.21.15.22:0/2723277413 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8011d20 con 0x7fe8080d1520 2023-07-06T01:19:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.784+0000 7fe80cd81700 1 -- 172.21.15.22:0/2723277413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8080d1520 msgr2=0x7fe80806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.784+0000 7fe80cd81700 1 --2- 172.21.15.22:0/2723277413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8080d1520 0x7fe80806c630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009840 tx=0x7fe7f8009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.784+0000 7fe80cd81700 1 -- 172.21.15.22:0/2723277413 shutdown_connections 2023-07-06T01:19:23.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.784+0000 7fe80cd81700 1 --2- 172.21.15.22:0/2723277413 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe80806b110 0x7fe8080cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.784+0000 7fe80cd81700 1 --2- 172.21.15.22:0/2723277413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8080d1520 0x7fe80806c630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.784+0000 7fe80cd81700 1 -- 172.21.15.22:0/2723277413 >> 172.21.15.22:0/2723277413 conn(0x7fe808066620 msgr2=0x7fe80809a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.784+0000 7fe80cd81700 1 -- 172.21.15.22:0/2723277413 shutdown_connections 2023-07-06T01:19:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.784+0000 7fe80cd81700 1 -- 172.21.15.22:0/2723277413 wait complete. 2023-07-06T01:19:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.786+0000 7fe80cd81700 1 Processor -- start 2023-07-06T01:19:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.786+0000 7fe80cd81700 1 -- start start 2023-07-06T01:19:23.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.786+0000 7fe80cd81700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe80806b110 0x7fe8080b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.786+0000 7fe80cd81700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8080d1520 0x7fe80814b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.786+0000 7fe80cd81700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe80815cd60 0x7fe8080cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.786+0000 7fe80cd81700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080b1f60 con 0x7fe80815cd60 2023-07-06T01:19:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.786+0000 7fe80cd81700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080cce50 con 0x7fe80806b110 2023-07-06T01:19:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.786+0000 7fe80cd81700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080b12e0 con 0x7fe8080d1520 2023-07-06T01:19:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe806d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe80815cd60 0x7fe8080cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:23.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe806d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe80815cd60 0x7fe8080cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45204/0 (socket says 172.21.15.22:45204) 2023-07-06T01:19:23.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe806d9d700 1 -- 172.21.15.22:0/1738400043 learned_addr learned my addr 172.21.15.22:0/1738400043 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:23.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe80659c700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe80806b110 0x7fe8080b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:23.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe805d9b700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8080d1520 0x7fe80814b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:23.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe806d9d700 1 -- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8080d1520 msgr2=0x7fe80814b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:23.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe806d9d700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8080d1520 0x7fe80814b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:23.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe806d9d700 1 -- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe80806b110 msgr2=0x7fe8080b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:23.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe806d9d700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe80806b110 0x7fe8080b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:23.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe806d9d700 1 -- 172.21.15.22:0/1738400043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f80094a0 con 0x7fe80815cd60 2023-07-06T01:19:23.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe80659c700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe80806b110 0x7fe8080b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:23.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe806d9d700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe80815cd60 0x7fe8080cc910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe800007e90 tx=0x7fe80000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:23.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe7f6ffd700 1 -- 172.21.15.22:0/1738400043 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80000ede0 con 0x7fe80815cd60 2023-07-06T01:19:23.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe7f6ffd700 1 -- 172.21.15.22:0/1738400043 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe800014420 con 0x7fe80815cd60 2023-07-06T01:19:23.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe80cd81700 1 -- 172.21.15.22:0/1738400043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8080b15c0 con 0x7fe80815cd60 2023-07-06T01:19:23.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe7f6ffd700 1 -- 172.21.15.22:0/1738400043 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8000147d0 con 0x7fe80815cd60 2023-07-06T01:19:23.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.787+0000 7fe80cd81700 1 -- 172.21.15.22:0/1738400043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8080b1b80 con 0x7fe80815cd60 2023-07-06T01:19:23.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.788+0000 7fe80cd81700 1 -- 172.21.15.22:0/1738400043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8080b20a0 con 0x7fe80815cd60 2023-07-06T01:19:23.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.789+0000 7fe7f6ffd700 1 -- 172.21.15.22:0/1738400043 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe800014930 con 0x7fe80815cd60 2023-07-06T01:19:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.789+0000 7fe7f6ffd700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe7e806ace0 0x7fe7e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.789+0000 7fe7f6ffd700 1 -- 172.21.15.22:0/1738400043 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fe800089900 con 0x7fe80815cd60 2023-07-06T01:19:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.789+0000 7fe80659c700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe7e806ace0 0x7fe7e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.790+0000 7fe80659c700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe7e806ace0 0x7fe7e806d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc000f60 tx=0x7fe7fc007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:23.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.794+0000 7fe7f6ffd700 1 -- 172.21.15.22:0/1738400043 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe8080b20a0 con 0x7fe80815cd60 2023-07-06T01:19:23.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:23.988+0000 7fe80cd81700 1 -- 172.21.15.22:0/1738400043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe8080b20a0 con 0x7fe80815cd60 2023-07-06T01:19:24.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.627+0000 7fe7f6ffd700 1 -- 172.21.15.22:0/1738400043 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe8080b20a0 con 0x7fe80815cd60 2023-07-06T01:19:24.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.630+0000 7fe80cd81700 1 -- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe7e806ace0 msgr2=0x7fe7e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:24.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.630+0000 7fe80cd81700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe7e806ace0 0x7fe7e806d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc000f60 tx=0x7fe7fc007ca0 comp rx=0 tx=0).stop 2023-07-06T01:19:24.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.630+0000 7fe80cd81700 1 -- 172.21.15.22:0/1738400043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe80815cd60 msgr2=0x7fe8080cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:24.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.630+0000 7fe80cd81700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe80815cd60 0x7fe8080cc910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe800007e90 tx=0x7fe80000e3c0 comp rx=0 tx=0).stop 2023-07-06T01:19:24.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.631+0000 7fe80cd81700 1 -- 172.21.15.22:0/1738400043 shutdown_connections 2023-07-06T01:19:24.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.631+0000 7fe80cd81700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe80815cd60 0x7fe8080cc910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:24.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.631+0000 7fe80cd81700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe7e806ace0 0x7fe7e806d190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:24.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.631+0000 7fe80cd81700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe80806b110 0x7fe8080b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:24.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.631+0000 7fe80cd81700 1 --2- 172.21.15.22:0/1738400043 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8080d1520 0x7fe80814b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:24.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.631+0000 7fe80cd81700 1 -- 172.21.15.22:0/1738400043 >> 172.21.15.22:0/1738400043 conn(0x7fe808066620 msgr2=0x7fe8080cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:24.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.631+0000 7fe80cd81700 1 -- 172.21.15.22:0/1738400043 shutdown_connections 2023-07-06T01:19:24.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.631+0000 7fe80cd81700 1 -- 172.21.15.22:0/1738400043 wait complete. 2023-07-06T01:19:24.643 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2023-07-06T01:19:24.659 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T01:19:24.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.786+0000 7f2cbb244700 1 Processor -- start 2023-07-06T01:19:24.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.786+0000 7f2cbb244700 1 -- start start 2023-07-06T01:19:24.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.787+0000 7f2cbb244700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cb40af2a0 0x7f2cb406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:24.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.787+0000 7f2cbb244700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2cb40cc320 0x7f2cb406a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:24.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.787+0000 7f2cbb244700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb40af670 con 0x7f2cb406a000 2023-07-06T01:19:24.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.787+0000 7f2cbb244700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb406a9e0 con 0x7f2cb40cc320 2023-07-06T01:19:24.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.787+0000 7f2cbb244700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb40cc7a0 con 0x7f2cb40af2a0 2023-07-06T01:19:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.787+0000 7f2cb97e1700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2cb40cc320 0x7f2cb406a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.787+0000 7f2cb3fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f2cb406a000 0x7f2cb40cbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35956/0 (socket says 172.21.15.22:35956) 2023-07-06T01:19:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.787+0000 7f2cb3fff700 1 -- 172.21.15.22:0/3051296802 learned_addr learned my addr 172.21.15.22:0/3051296802 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:24.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.787+0000 7f2cb97e1700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2cb40cc320 0x7f2cb406a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45214/0 (socket says 172.21.15.22:45214) 2023-07-06T01:19:24.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.788+0000 7f2cb8fe0700 1 --2- 172.21.15.22:0/3051296802 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cb40af2a0 0x7f2cb406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:24.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.788+0000 7f2cb97e1700 1 -- 172.21.15.22:0/3051296802 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cb40af2a0 msgr2=0x7f2cb406caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:24.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.788+0000 7f2cb97e1700 1 --2- 172.21.15.22:0/3051296802 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cb40af2a0 0x7f2cb406caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:24.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.788+0000 7f2cb97e1700 1 -- 172.21.15.22:0/3051296802 >> v1:172.21.15.22:6789/0 conn(0x7f2cb406a000 legacy=0x7f2cb40cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:24.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.788+0000 7f2cb97e1700 1 -- 172.21.15.22:0/3051296802 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb406d030 con 0x7f2cb40cc320 2023-07-06T01:19:24.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.788+0000 7f2cb37fe700 1 -- 172.21.15.22:0/3051296802 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135215810 0 0) 0x7f2cb40af670 con 0x7f2cb406a000 2023-07-06T01:19:24.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.788+0000 7f2cb97e1700 1 --2- 172.21.15.22:0/3051296802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb40cc320 0x7f2cb406a4a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009d90 tx=0x7f2cac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64048f2fbe986111 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:24.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.788+0000 7f2cb37fe700 1 -- 172.21.15.22:0/3051296802 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00e070 con 0x7f2cb40cc320 2023-07-06T01:19:24.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.788+0000 7f2cb37fe700 1 -- 172.21.15.22:0/3051296802 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac00bbd0 con 0x7f2cb40cc320 2023-07-06T01:19:24.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.788+0000 7f2cb37fe700 1 -- 172.21.15.22:0/3051296802 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac012430 con 0x7f2cb40cc320 2023-07-06T01:19:24.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.789+0000 7f2cbb244700 1 -- 172.21.15.22:0/3051296802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb40cc320 msgr2=0x7f2cb406a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:24.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.789+0000 7f2cbb244700 1 --2- 172.21.15.22:0/3051296802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb40cc320 0x7f2cb406a4a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009d90 tx=0x7f2cac00b330 comp rx=0 tx=0).stop 2023-07-06T01:19:24.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.789+0000 7f2cbb244700 1 -- 172.21.15.22:0/3051296802 shutdown_connections 2023-07-06T01:19:24.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.789+0000 7f2cbb244700 1 --2- 172.21.15.22:0/3051296802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb40cc320 0x7f2cb406a4a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:24.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.789+0000 7f2cbb244700 1 --2- 172.21.15.22:0/3051296802 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cb40af2a0 0x7f2cb406caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:24.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.789+0000 7f2cbb244700 1 -- 172.21.15.22:0/3051296802 >> 172.21.15.22:0/3051296802 conn(0x7f2cb40b1890 msgr2=0x7f2cb40b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:24.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.789+0000 7f2cbb244700 1 -- 172.21.15.22:0/3051296802 shutdown_connections 2023-07-06T01:19:24.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.789+0000 7f2cbb244700 1 -- 172.21.15.22:0/3051296802 wait complete. 2023-07-06T01:19:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.790+0000 7f2cbb244700 1 Processor -- start 2023-07-06T01:19:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.790+0000 7f2cbb244700 1 -- start start 2023-07-06T01:19:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cbb244700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cb406a000 0x7f2cb40b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cbb244700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb40af2a0 0x7f2cb415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cbb244700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cb40cc320 0x7f2cb40b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cbb244700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb415ba10 con 0x7f2cb40af2a0 2023-07-06T01:19:24.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cbb244700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb40b4df0 con 0x7f2cb40cc320 2023-07-06T01:19:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cbb244700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb40b4420 con 0x7f2cb406a000 2023-07-06T01:19:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cb3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb40af2a0 0x7f2cb415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cb3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb40af2a0 0x7f2cb415ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45218/0 (socket says 172.21.15.22:45218) 2023-07-06T01:19:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cb3fff700 1 -- 172.21.15.22:0/1881680927 learned_addr learned my addr 172.21.15.22:0/1881680927 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cb8fe0700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cb406a000 0x7f2cb40b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cb97e1700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cb40cc320 0x7f2cb40b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cb3fff700 1 -- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cb406a000 msgr2=0x7f2cb40b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:24.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cb3fff700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cb406a000 0x7f2cb40b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:24.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cb3fff700 1 -- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cb40cc320 msgr2=0x7f2cb40b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:24.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cb3fff700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cb40cc320 0x7f2cb40b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:24.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cb3fff700 1 -- 172.21.15.22:0/1881680927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac009a70 con 0x7f2cb40af2a0 2023-07-06T01:19:24.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.791+0000 7f2cb3fff700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb40af2a0 0x7f2cb415ae30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00011e0 tx=0x7f2ca000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:24.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.792+0000 7f2cb17fa700 1 -- 172.21.15.22:0/1881680927 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0004c80 con 0x7f2cb40af2a0 2023-07-06T01:19:24.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.792+0000 7f2cb17fa700 1 -- 172.21.15.22:0/1881680927 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca0004e00 con 0x7f2cb40af2a0 2023-07-06T01:19:24.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.792+0000 7f2cbb244700 1 -- 172.21.15.22:0/1881680927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb40b3810 con 0x7f2cb40af2a0 2023-07-06T01:19:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.792+0000 7f2cb17fa700 1 -- 172.21.15.22:0/1881680927 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0007310 con 0x7f2cb40af2a0 2023-07-06T01:19:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.792+0000 7f2cbb244700 1 -- 172.21.15.22:0/1881680927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb40b40b0 con 0x7f2cb40af2a0 2023-07-06T01:19:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.793+0000 7f2cbb244700 1 -- 172.21.15.22:0/1881680927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb415ca90 con 0x7f2cb40af2a0 2023-07-06T01:19:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.793+0000 7f2cb17fa700 1 -- 172.21.15.22:0/1881680927 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2ca0007470 con 0x7f2cb40af2a0 2023-07-06T01:19:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.793+0000 7f2cb17fa700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2c9806ace0 0x7f2c9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.794+0000 7f2cb17fa700 1 -- 172.21.15.22:0/1881680927 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f2ca00890a0 con 0x7f2cb40af2a0 2023-07-06T01:19:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.794+0000 7f2cb8fe0700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2c9806ace0 0x7f2c9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:24.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.794+0000 7f2cb8fe0700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2c9806ace0 0x7f2c9806d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800ab60 tx=0x7f2ca8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:24.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:24.798+0000 7f2cb17fa700 1 -- 172.21.15.22:0/1881680927 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2cb415ca90 con 0x7f2cb40af2a0 2023-07-06T01:19:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.040+0000 7f2cbb244700 1 -- 172.21.15.22:0/1881680927 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2cb415ca90 con 0x7f2cb40af2a0 2023-07-06T01:19:25.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.630+0000 7f2cb17fa700 1 -- 172.21.15.22:0/1881680927 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2cb415ca90 con 0x7f2cb40af2a0 2023-07-06T01:19:25.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.636+0000 7f2cbb244700 1 -- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2c9806ace0 msgr2=0x7f2c9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:25.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.636+0000 7f2cbb244700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2c9806ace0 0x7f2c9806d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800ab60 tx=0x7f2ca8008040 comp rx=0 tx=0).stop 2023-07-06T01:19:25.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.636+0000 7f2cbb244700 1 -- 172.21.15.22:0/1881680927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb40af2a0 msgr2=0x7f2cb415ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:25.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.636+0000 7f2cbb244700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb40af2a0 0x7f2cb415ae30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00011e0 tx=0x7f2ca000ca30 comp rx=0 tx=0).stop 2023-07-06T01:19:25.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.638+0000 7f2cbb244700 1 -- 172.21.15.22:0/1881680927 shutdown_connections 2023-07-06T01:19:25.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.638+0000 7f2cbb244700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2c9806ace0 0x7f2c9806d190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:25.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.638+0000 7f2cbb244700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cb40cc320 0x7f2cb40b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:25.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.638+0000 7f2cbb244700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cb406a000 0x7f2cb40b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:25.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.638+0000 7f2cbb244700 1 --2- 172.21.15.22:0/1881680927 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2cb40af2a0 0x7f2cb415ae30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:25.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.638+0000 7f2cbb244700 1 -- 172.21.15.22:0/1881680927 >> 172.21.15.22:0/1881680927 conn(0x7f2cb40b1890 msgr2=0x7f2cb406dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:25.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.639+0000 7f2cbb244700 1 -- 172.21.15.22:0/1881680927 shutdown_connections 2023-07-06T01:19:25.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.640+0000 7f2cbb244700 1 -- 172.21.15.22:0/1881680927 wait complete. 2023-07-06T01:19:25.659 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 3 and data pool 4 2023-07-06T01:19:25.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:25.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.799+0000 7ffad29e7700 1 Processor -- start 2023-07-06T01:19:25.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.799+0000 7ffad29e7700 1 -- start start 2023-07-06T01:19:25.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.800+0000 7ffad29e7700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ffacc06a710 0x7ffacc06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:25.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.800+0000 7ffad29e7700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffacc0af670 con 0x7ffacc0af2a0 2023-07-06T01:19:25.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.800+0000 7ffad29e7700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffacc06d030 con 0x7ffacc0cbc10 2023-07-06T01:19:25.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.800+0000 7ffad29e7700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffacc0cc780 con 0x7ffacc06a710 2023-07-06T01:19:25.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.800+0000 7ffacb7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ffacc06a710 0x7ffacc06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:25.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.800+0000 7ffacbfff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ffacc0af2a0 0x7ffacc06a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35968/0 (socket says 172.21.15.22:35968) 2023-07-06T01:19:25.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.800+0000 7ffacbfff700 1 -- 172.21.15.22:0/2113203136 learned_addr learned my addr 172.21.15.22:0/2113203136 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:25.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.800+0000 7ffacb7fe700 1 -- 172.21.15.22:0/2113203136 >> v1:172.21.15.103:6789/0 conn(0x7ffacc0cbc10 legacy=0x7ffacc0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:25.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.800+0000 7ffacb7fe700 1 -- 172.21.15.22:0/2113203136 >> v1:172.21.15.22:6789/0 conn(0x7ffacc0af2a0 legacy=0x7ffacc06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:25.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.800+0000 7ffacaffd700 1 -- 172.21.15.22:0/2113203136 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083316826 0 0) 0x7ffacc0af670 con 0x7ffacc0af2a0 2023-07-06T01:19:25.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.800+0000 7ffacb7fe700 1 -- 172.21.15.22:0/2113203136 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc0cd560 con 0x7ffacc06a710 2023-07-06T01:19:25.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.801+0000 7ffacb7fe700 1 --2- 172.21.15.22:0/2113203136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffacc06a710 0x7ffacc06caf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009840 tx=0x7ffab4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc8d9d7a38a49b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:25.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.801+0000 7ffacaffd700 1 -- 172.21.15.22:0/2113203136 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4004230 con 0x7ffacc06a710 2023-07-06T01:19:25.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.801+0000 7ffacaffd700 1 -- 172.21.15.22:0/2113203136 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab4004390 con 0x7ffacc06a710 2023-07-06T01:19:25.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.801+0000 7ffacaffd700 1 -- 172.21.15.22:0/2113203136 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab40036a0 con 0x7ffacc06a710 2023-07-06T01:19:25.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.801+0000 7ffad29e7700 1 -- 172.21.15.22:0/2113203136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffacc06a710 msgr2=0x7ffacc06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:25.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.801+0000 7ffad29e7700 1 --2- 172.21.15.22:0/2113203136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffacc06a710 0x7ffacc06caf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009840 tx=0x7ffab4009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:25.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.801+0000 7ffad29e7700 1 -- 172.21.15.22:0/2113203136 shutdown_connections 2023-07-06T01:19:25.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.801+0000 7ffad29e7700 1 --2- 172.21.15.22:0/2113203136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffacc06a710 0x7ffacc06caf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:25.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.801+0000 7ffad29e7700 1 -- 172.21.15.22:0/2113203136 >> 172.21.15.22:0/2113203136 conn(0x7ffacc0b1890 msgr2=0x7ffacc0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:25.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.801+0000 7ffad29e7700 1 -- 172.21.15.22:0/2113203136 shutdown_connections 2023-07-06T01:19:25.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.801+0000 7ffad29e7700 1 -- 172.21.15.22:0/2113203136 wait complete. 2023-07-06T01:19:25.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.802+0000 7ffad29e7700 1 Processor -- start 2023-07-06T01:19:25.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.803+0000 7ffad29e7700 1 -- start start 2023-07-06T01:19:25.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.803+0000 7ffad29e7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffacc06a710 0x7ffacc151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:25.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.803+0000 7ffad29e7700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffacc0af2a0 0x7ffacc0b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:25.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.803+0000 7ffad29e7700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffacc0cbc10 0x7ffacc158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:25.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.803+0000 7ffad29e7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc159ad0 con 0x7ffacc06a710 2023-07-06T01:19:25.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.803+0000 7ffad29e7700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc151ca0 con 0x7ffacc0cbc10 2023-07-06T01:19:25.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.803+0000 7ffad29e7700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc159020 con 0x7ffacc0af2a0 2023-07-06T01:19:25.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffacbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffacc06a710 0x7ffacc151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:25.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffacbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffacc06a710 0x7ffacc151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45230/0 (socket says 172.21.15.22:45230) 2023-07-06T01:19:25.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffacbfff700 1 -- 172.21.15.22:0/1948202936 learned_addr learned my addr 172.21.15.22:0/1948202936 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:25.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffacb7fe700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffacc0af2a0 0x7ffacc0b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:25.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffad0f84700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffacc0cbc10 0x7ffacc158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:25.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffacbfff700 1 -- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffacc0af2a0 msgr2=0x7ffacc0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:25.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffacbfff700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffacc0af2a0 0x7ffacc0b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:25.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffacbfff700 1 -- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffacc0cbc10 msgr2=0x7ffacc158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffacbfff700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffacc0cbc10 0x7ffacc158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffacbfff700 1 -- 172.21.15.22:0/1948202936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab40094a0 con 0x7ffacc06a710 2023-07-06T01:19:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffad0f84700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffacc0cbc10 0x7ffacc158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffacb7fe700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffacc0af2a0 0x7ffacc0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffacbfff700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffacc06a710 0x7ffacc151390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0011e0 tx=0x7ffabc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffac8ff9700 1 -- 172.21.15.22:0/1948202936 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc004c80 con 0x7ffacc06a710 2023-07-06T01:19:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffac8ff9700 1 -- 172.21.15.22:0/1948202936 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffabc004e00 con 0x7ffacc06a710 2023-07-06T01:19:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffad29e7700 1 -- 172.21.15.22:0/1948202936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffacc15e110 con 0x7ffacc06a710 2023-07-06T01:19:25.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffac8ff9700 1 -- 172.21.15.22:0/1948202936 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc007310 con 0x7ffacc06a710 2023-07-06T01:19:25.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.804+0000 7ffad29e7700 1 -- 172.21.15.22:0/1948202936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffacc159510 con 0x7ffacc06a710 2023-07-06T01:19:25.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.805+0000 7ffad29e7700 1 -- 172.21.15.22:0/1948202936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffacc15ca90 con 0x7ffacc06a710 2023-07-06T01:19:25.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.806+0000 7ffac8ff9700 1 -- 172.21.15.22:0/1948202936 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffabc007470 con 0x7ffacc06a710 2023-07-06T01:19:25.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.806+0000 7ffac8ff9700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ffaac06ace0 0x7ffaac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:25.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.806+0000 7ffac8ff9700 1 -- 172.21.15.22:0/1948202936 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffabc0895b0 con 0x7ffacc06a710 2023-07-06T01:19:25.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.807+0000 7ffacb7fe700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ffaac06ace0 0x7ffaac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:25.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.807+0000 7ffacb7fe700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ffaac06ace0 0x7ffaac06d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009840 tx=0x7ffab4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:25.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:25.810+0000 7ffac8ff9700 1 -- 172.21.15.22:0/1948202936 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ffabc0547d0 con 0x7ffacc06a710 2023-07-06T01:19:26.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.015+0000 7ffad29e7700 1 -- 172.21.15.22:0/1948202936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffacc15ca90 con 0x7ffacc06a710 2023-07-06T01:19:26.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.016+0000 7ffac8ff9700 1 -- 172.21.15.22:0/1948202936 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18222 (secure 0 0 0) 0x7ffacc15ca90 con 0x7ffacc06a710 2023-07-06T01:19:26.018 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:26.019 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:25.041865+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:26.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.018+0000 7ffad29e7700 1 -- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ffaac06ace0 msgr2=0x7ffaac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:26.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.018+0000 7ffad29e7700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ffaac06ace0 0x7ffaac06d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009840 tx=0x7ffab4008030 comp rx=0 tx=0).stop 2023-07-06T01:19:26.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.018+0000 7ffad29e7700 1 -- 172.21.15.22:0/1948202936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffacc06a710 msgr2=0x7ffacc151390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:26.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.018+0000 7ffad29e7700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffacc06a710 0x7ffacc151390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0011e0 tx=0x7ffabc00cae0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.018+0000 7ffad29e7700 1 -- 172.21.15.22:0/1948202936 shutdown_connections 2023-07-06T01:19:26.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.018+0000 7ffad29e7700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ffaac06ace0 0x7ffaac06d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.018+0000 7ffad29e7700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffacc0cbc10 0x7ffacc158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.018+0000 7ffad29e7700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffacc06a710 0x7ffacc151390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.018+0000 7ffad29e7700 1 --2- 172.21.15.22:0/1948202936 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffacc0af2a0 0x7ffacc0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.018+0000 7ffad29e7700 1 -- 172.21.15.22:0/1948202936 >> 172.21.15.22:0/1948202936 conn(0x7ffacc0b1890 msgr2=0x7ffacc0ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:26.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.018+0000 7ffad29e7700 1 -- 172.21.15.22:0/1948202936 shutdown_connections 2023-07-06T01:19:26.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.018+0000 7ffad29e7700 1 -- 172.21.15.22:0/1948202936 wait complete. 2023-07-06T01:19:26.032 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:26.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.155+0000 7fadde9bd700 1 Processor -- start 2023-07-06T01:19:26.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.155+0000 7fadde9bd700 1 -- start start 2023-07-06T01:19:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.156+0000 7fadde9bd700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadd815a710 0x7fadd80ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.156+0000 7fadde9bd700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fadd80aecb0 0x7fadd80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.156+0000 7fadde9bd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fadd80ceda0 0x7fadd815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.156+0000 7fadde9bd700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd80cf220 con 0x7fadd80ceda0 2023-07-06T01:19:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.156+0000 7fadde9bd700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd80af270 con 0x7fadd80aecb0 2023-07-06T01:19:26.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.156+0000 7fadde9bd700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80af550 con 0x7fadd815a710 2023-07-06T01:19:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.156+0000 7faddcf5a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fadd80ceda0 0x7fadd815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.156+0000 7faddcf5a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fadd80ceda0 0x7fadd815e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45240/0 (socket says 172.21.15.22:45240) 2023-07-06T01:19:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.156+0000 7faddcf5a700 1 -- 172.21.15.22:0/3806189847 learned_addr learned my addr 172.21.15.22:0/3806189847 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.156+0000 7fadd77fe700 1 --2- 172.21.15.22:0/3806189847 >> v2:172.21.15.103:3300/0 conn(0x7fadd80aecb0 0x7fadd80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.156+0000 7fadd7fff700 1 --2- 172.21.15.22:0/3806189847 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadd815a710 0x7fadd80ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.157+0000 7faddcf5a700 1 -- 172.21.15.22:0/3806189847 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadd815a710 msgr2=0x7fadd80ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:26.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.157+0000 7faddcf5a700 1 --2- 172.21.15.22:0/3806189847 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadd815a710 0x7fadd80ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.157+0000 7faddcf5a700 1 -- 172.21.15.22:0/3806189847 >> v2:172.21.15.103:3300/0 conn(0x7fadd80aecb0 msgr2=0x7fadd80ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:26.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.157+0000 7faddcf5a700 1 --2- 172.21.15.22:0/3806189847 >> v2:172.21.15.103:3300/0 conn(0x7fadd80aecb0 0x7fadd80ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.157+0000 7faddcf5a700 1 -- 172.21.15.22:0/3806189847 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd80af830 con 0x7fadd80ceda0 2023-07-06T01:19:26.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.157+0000 7faddcf5a700 1 --2- 172.21.15.22:0/3806189847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadd80ceda0 0x7fadd815e0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fadd80716d0 tx=0x7fadd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1984ea14646228b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:26.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.157+0000 7fadd6ffd700 1 -- 172.21.15.22:0/3806189847 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000e070 con 0x7fadd80ceda0 2023-07-06T01:19:26.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.157+0000 7fadd6ffd700 1 -- 172.21.15.22:0/3806189847 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd000b910 con 0x7fadd80ceda0 2023-07-06T01:19:26.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.157+0000 7fadd6ffd700 1 -- 172.21.15.22:0/3806189847 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000bcc0 con 0x7fadd80ceda0 2023-07-06T01:19:26.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.158+0000 7fadde9bd700 1 -- 172.21.15.22:0/3806189847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadd80ceda0 msgr2=0x7fadd815e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:26.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.158+0000 7fadde9bd700 1 --2- 172.21.15.22:0/3806189847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadd80ceda0 0x7fadd815e0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fadd80716d0 tx=0x7fadd000b330 comp rx=0 tx=0).stop 2023-07-06T01:19:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.158+0000 7fadde9bd700 1 -- 172.21.15.22:0/3806189847 shutdown_connections 2023-07-06T01:19:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.158+0000 7fadde9bd700 1 --2- 172.21.15.22:0/3806189847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadd80ceda0 0x7fadd815e0d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.158+0000 7fadde9bd700 1 --2- 172.21.15.22:0/3806189847 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadd815a710 0x7fadd80ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.158+0000 7fadde9bd700 1 --2- 172.21.15.22:0/3806189847 >> v2:172.21.15.103:3300/0 conn(0x7fadd80aecb0 0x7fadd80ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.158+0000 7fadde9bd700 1 -- 172.21.15.22:0/3806189847 >> 172.21.15.22:0/3806189847 conn(0x7fadd80999f0 msgr2=0x7fadd8099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.158+0000 7fadde9bd700 1 -- 172.21.15.22:0/3806189847 shutdown_connections 2023-07-06T01:19:26.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.158+0000 7fadde9bd700 1 -- 172.21.15.22:0/3806189847 wait complete. 2023-07-06T01:19:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.159+0000 7fadde9bd700 1 Processor -- start 2023-07-06T01:19:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.160+0000 7fadde9bd700 1 -- start start 2023-07-06T01:19:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.160+0000 7fadde9bd700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadd80aecb0 0x7fadd814e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.160+0000 7fadde9bd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadd80ceda0 0x7fadd80c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.160+0000 7fadde9bd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadd815a710 0x7fadd80c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.160+0000 7fadde9bd700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80c4580 con 0x7fadd80ceda0 2023-07-06T01:19:26.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.160+0000 7fadde9bd700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80c9120 con 0x7fadd815a710 2023-07-06T01:19:26.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.160+0000 7fadde9bd700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd806c920 con 0x7fadd80aecb0 2023-07-06T01:19:26.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadd77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadd80ceda0 0x7fadd80c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:26.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadd77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadd80ceda0 0x7fadd80c7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45244/0 (socket says 172.21.15.22:45244) 2023-07-06T01:19:26.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadd77fe700 1 -- 172.21.15.22:0/882794585 learned_addr learned my addr 172.21.15.22:0/882794585 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:26.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadd7fff700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadd80aecb0 0x7fadd814e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:26.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7faddcf5a700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadd815a710 0x7fadd80c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:26.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadd77fe700 1 -- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadd80aecb0 msgr2=0x7fadd814e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:26.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadd77fe700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadd80aecb0 0x7fadd814e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadd77fe700 1 -- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadd815a710 msgr2=0x7fadd80c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:26.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadd77fe700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadd815a710 0x7fadd80c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadd77fe700 1 -- 172.21.15.22:0/882794585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd0009b30 con 0x7fadd80ceda0 2023-07-06T01:19:26.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadd77fe700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadd80ceda0 0x7fadd80c7050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00e9b0 tx=0x7fadcc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:26.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadd4ff9700 1 -- 172.21.15.22:0/882794585 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc009710 con 0x7fadd80ceda0 2023-07-06T01:19:26.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadd4ff9700 1 -- 172.21.15.22:0/882794585 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadcc00ee50 con 0x7fadd80ceda0 2023-07-06T01:19:26.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadde9bd700 1 -- 172.21.15.22:0/882794585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd806cc00 con 0x7fadd80ceda0 2023-07-06T01:19:26.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadd4ff9700 1 -- 172.21.15.22:0/882794585 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc00ca50 con 0x7fadd80ceda0 2023-07-06T01:19:26.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.161+0000 7fadde9bd700 1 -- 172.21.15.22:0/882794585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd80c8db0 con 0x7fadd80ceda0 2023-07-06T01:19:26.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.162+0000 7fadde9bd700 1 -- 172.21.15.22:0/882794585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd80744b0 con 0x7fadd80ceda0 2023-07-06T01:19:26.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.163+0000 7fadd4ff9700 1 -- 172.21.15.22:0/882794585 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fadcc00cbb0 con 0x7fadd80ceda0 2023-07-06T01:19:26.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.163+0000 7fadd4ff9700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fadc406ace0 0x7fadc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:26.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.163+0000 7fadd4ff9700 1 -- 172.21.15.22:0/882794585 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fadcc089470 con 0x7fadd80ceda0 2023-07-06T01:19:26.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.163+0000 7fadd7fff700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fadc406ace0 0x7fadc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:26.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.164+0000 7fadd7fff700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fadc406ace0 0x7fadc406d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009ac0 tx=0x7fadc8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:26.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.167+0000 7fadd4ff9700 1 -- 172.21.15.22:0/882794585 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fadcc057fb0 con 0x7fadd80ceda0 2023-07-06T01:19:26.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.366+0000 7fadde9bd700 1 -- 172.21.15.22:0/882794585 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadd80d1970 con 0x7fadd80ceda0 2023-07-06T01:19:26.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.368+0000 7fadd4ff9700 1 -- 172.21.15.22:0/882794585 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18222 (secure 0 0 0) 0x7fadcc00ce60 con 0x7fadd80ceda0 2023-07-06T01:19:26.370 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:26.370 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:25.041865+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:26.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.370+0000 7fadde9bd700 1 -- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fadc406ace0 msgr2=0x7fadc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:26.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.370+0000 7fadde9bd700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fadc406ace0 0x7fadc406d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009ac0 tx=0x7fadc8009210 comp rx=0 tx=0).stop 2023-07-06T01:19:26.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.370+0000 7fadde9bd700 1 -- 172.21.15.22:0/882794585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadd80ceda0 msgr2=0x7fadd80c7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.370+0000 7fadde9bd700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadd80ceda0 0x7fadd80c7050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00e9b0 tx=0x7fadcc00ecc0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.370+0000 7fadde9bd700 1 -- 172.21.15.22:0/882794585 shutdown_connections 2023-07-06T01:19:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.370+0000 7fadde9bd700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fadc406ace0 0x7fadc406d190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.370+0000 7fadde9bd700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadd815a710 0x7fadd80c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.370+0000 7fadde9bd700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadd80aecb0 0x7fadd814e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.370+0000 7fadde9bd700 1 --2- 172.21.15.22:0/882794585 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fadd80ceda0 0x7fadd80c7050 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.370+0000 7fadde9bd700 1 -- 172.21.15.22:0/882794585 >> 172.21.15.22:0/882794585 conn(0x7fadd80999f0 msgr2=0x7fadd80cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.370+0000 7fadde9bd700 1 -- 172.21.15.22:0/882794585 shutdown_connections 2023-07-06T01:19:26.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.370+0000 7fadde9bd700 1 -- 172.21.15.22:0/882794585 wait complete. 2023-07-06T01:19:26.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T01:19:26.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.510+0000 7f8da1b9d700 1 Processor -- start 2023-07-06T01:19:26.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.510+0000 7f8da1b9d700 1 -- start start 2023-07-06T01:19:26.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.510+0000 7f8da1b9d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d9c0b37f0 0x7f8d9c160050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:26.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8da1b9d700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9c06a0a0 con 0x7f8d9c071df0 2023-07-06T01:19:26.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8da1b9d700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9c160630 con 0x7f8d9c0700e0 2023-07-06T01:19:26.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8da1b9d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c160910 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8d9bfff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f8d9c071df0 0x7f8d9c0b3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:35994/0 (socket says 172.21.15.22:35994) 2023-07-06T01:19:26.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8d9bfff700 1 -- 172.21.15.22:0/3029049143 learned_addr learned my addr 172.21.15.22:0/3029049143 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:26.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8da0b9b700 1 --2- 172.21.15.22:0/3029049143 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d9c0b37f0 0x7f8d9c160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:26.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8d9b7fe700 1 -- 172.21.15.22:0/3029049143 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407547281 0 0) 0x7f8d9c06a0a0 con 0x7f8d9c071df0 2023-07-06T01:19:26.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8d9b7fe700 1 -- 172.21.15.22:0/3029049143 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d8c0035b0 con 0x7f8d9c071df0 2023-07-06T01:19:26.515 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8d9b7fe700 1 -- 172.21.15.22:0/3029049143 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405139575 0 0) 0x7f8d9c160630 con 0x7f8d9c0700e0 2023-07-06T01:19:26.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8d9b7fe700 1 -- 172.21.15.22:0/3029049143 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d9c06a0a0 con 0x7f8d9c0700e0 2023-07-06T01:19:26.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8da0b9b700 1 -- 172.21.15.22:0/3029049143 >> v1:172.21.15.103:6789/0 conn(0x7f8d9c0700e0 legacy=0x7f8d9c06a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:26.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8da0b9b700 1 -- 172.21.15.22:0/3029049143 >> v1:172.21.15.22:6789/0 conn(0x7f8d9c071df0 legacy=0x7f8d9c0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:26.516 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8da0b9b700 1 -- 172.21.15.22:0/3029049143 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c160bf0 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.511+0000 7f8da0b9b700 1 --2- 172.21.15.22:0/3029049143 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d9c0b37f0 0x7f8d9c160050 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009870 tx=0x7f8d90009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdf46d177c7d5f54 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:26.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.512+0000 7f8d9b7fe700 1 -- 172.21.15.22:0/3029049143 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90011830 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.512+0000 7f8d9b7fe700 1 -- 172.21.15.22:0/3029049143 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d90011990 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.512+0000 7f8d9b7fe700 1 -- 172.21.15.22:0/3029049143 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90011d20 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.512+0000 7f8da1b9d700 1 -- 172.21.15.22:0/3029049143 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d9c0b37f0 msgr2=0x7f8d9c160050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:26.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.512+0000 7f8da1b9d700 1 --2- 172.21.15.22:0/3029049143 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d9c0b37f0 0x7f8d9c160050 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009870 tx=0x7f8d90009b80 comp rx=0 tx=0).stop 2023-07-06T01:19:26.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.512+0000 7f8da1b9d700 1 -- 172.21.15.22:0/3029049143 shutdown_connections 2023-07-06T01:19:26.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.512+0000 7f8da1b9d700 1 --2- 172.21.15.22:0/3029049143 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d9c0b37f0 0x7f8d9c160050 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.512+0000 7f8da1b9d700 1 -- 172.21.15.22:0/3029049143 >> 172.21.15.22:0/3029049143 conn(0x7f8d9c0cf1f0 msgr2=0x7f8d9c0cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:26.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.512+0000 7f8da1b9d700 1 -- 172.21.15.22:0/3029049143 shutdown_connections 2023-07-06T01:19:26.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.512+0000 7f8da1b9d700 1 -- 172.21.15.22:0/3029049143 wait complete. 2023-07-06T01:19:26.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.513+0000 7f8da1b9d700 1 Processor -- start 2023-07-06T01:19:26.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.513+0000 7f8da1b9d700 1 -- start start 2023-07-06T01:19:26.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.514+0000 7f8da1b9d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d9c0700e0 0x7f8d9c14e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:26.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.514+0000 7f8da1b9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d9c071df0 0x7f8d9c1e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:26.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.514+0000 7f8da1b9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d9c0b37f0 0x7f8d9c13aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:26.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.514+0000 7f8da1b9d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1e8b90 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.514+0000 7f8da1b9d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1e8350 con 0x7f8d9c071df0 2023-07-06T01:19:26.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.514+0000 7f8da1b9d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1e85d0 con 0x7f8d9c0700e0 2023-07-06T01:19:26.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.514+0000 7f8da139c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d9c0b37f0 0x7f8d9c13aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:26.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.514+0000 7f8da139c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d9c0b37f0 0x7f8d9c13aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45260/0 (socket says 172.21.15.22:45260) 2023-07-06T01:19:26.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.514+0000 7f8da139c700 1 -- 172.21.15.22:0/88839480 learned_addr learned my addr 172.21.15.22:0/88839480 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:26.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.515+0000 7f8d9bfff700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d9c071df0 0x7f8d9c1e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:26.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.515+0000 7f8da0b9b700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d9c0700e0 0x7f8d9c14e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:26.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.515+0000 7f8da139c700 1 -- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d9c0700e0 msgr2=0x7f8d9c14e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:26.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.515+0000 7f8da139c700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d9c0700e0 0x7f8d9c14e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.515+0000 7f8da139c700 1 -- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d9c071df0 msgr2=0x7f8d9c1e7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:26.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.515+0000 7f8da139c700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d9c071df0 0x7f8d9c1e7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:26.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.515+0000 7f8da139c700 1 -- 172.21.15.22:0/88839480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d900094a0 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.515+0000 7f8da139c700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d9c0b37f0 0x7f8d9c13aaa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400cbf0 tx=0x7f8d9400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:26.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.515+0000 7f8d997fa700 1 -- 172.21.15.22:0/88839480 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94003c00 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.515+0000 7f8d997fa700 1 -- 172.21.15.22:0/88839480 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d94003d60 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.515+0000 7f8da1b9d700 1 -- 172.21.15.22:0/88839480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c1e88b0 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.515+0000 7f8d997fa700 1 -- 172.21.15.22:0/88839480 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400aa60 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.515+0000 7f8da1b9d700 1 -- 172.21.15.22:0/88839480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c15cea0 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.516+0000 7f8da1b9d700 1 -- 172.21.15.22:0/88839480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9c04eb90 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.516+0000 7f8d997fa700 1 -- 172.21.15.22:0/88839480 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d9400abc0 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.517+0000 7f8d997fa700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8d8006ace0 0x7f8d8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:26.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.517+0000 7f8d997fa700 1 -- 172.21.15.22:0/88839480 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8d940893c0 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.517+0000 7f8da0b9b700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8d8006ace0 0x7f8d8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:26.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.518+0000 7f8da0b9b700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8d8006ace0 0x7f8d8006d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d900097c0 tx=0x7f8d9001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:26.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.521+0000 7f8d997fa700 1 -- 172.21.15.22:0/88839480 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8d94054500 con 0x7f8d9c0b37f0 2023-07-06T01:19:26.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:26.773+0000 7f8da1b9d700 1 -- 172.21.15.22:0/88839480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8d9c06e880 con 0x7f8d9c0b37f0 2023-07-06T01:19:27.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.650+0000 7f8d997fa700 1 -- 172.21.15.22:0/88839480 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8d94057b20 con 0x7f8d9c0b37f0 2023-07-06T01:19:27.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.653+0000 7f8da1b9d700 1 -- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8d8006ace0 msgr2=0x7f8d8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:27.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.653+0000 7f8da1b9d700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8d8006ace0 0x7f8d8006d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d900097c0 tx=0x7f8d9001c040 comp rx=0 tx=0).stop 2023-07-06T01:19:27.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.653+0000 7f8da1b9d700 1 -- 172.21.15.22:0/88839480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d9c0b37f0 msgr2=0x7f8d9c13aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:27.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.653+0000 7f8da1b9d700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d9c0b37f0 0x7f8d9c13aaa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400cbf0 tx=0x7f8d9400cf00 comp rx=0 tx=0).stop 2023-07-06T01:19:27.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.653+0000 7f8da1b9d700 1 -- 172.21.15.22:0/88839480 shutdown_connections 2023-07-06T01:19:27.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.653+0000 7f8da1b9d700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d9c0b37f0 0x7f8d9c13aaa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:27.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.653+0000 7f8da1b9d700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d9c0700e0 0x7f8d9c14e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:27.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.653+0000 7f8da1b9d700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8d8006ace0 0x7f8d8006d190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:27.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.653+0000 7f8da1b9d700 1 --2- 172.21.15.22:0/88839480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d9c071df0 0x7f8d9c1e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:27.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.653+0000 7f8da1b9d700 1 -- 172.21.15.22:0/88839480 >> 172.21.15.22:0/88839480 conn(0x7f8d9c0cf1f0 msgr2=0x7f8d9c06c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:27.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.653+0000 7f8da1b9d700 1 -- 172.21.15.22:0/88839480 shutdown_connections 2023-07-06T01:19:27.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.653+0000 7f8da1b9d700 1 -- 172.21.15.22:0/88839480 wait complete. 2023-07-06T01:19:27.670 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T01:19:27.670 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T01:19:27.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.804+0000 7f61be5fe700 1 Processor -- start 2023-07-06T01:19:27.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.804+0000 7f61be5fe700 1 -- start start 2023-07-06T01:19:27.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.805+0000 7f61be5fe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f61b80af2a0 0x7f61b806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:27.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.805+0000 7f61be5fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b806a000 0x7f61b806a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:27.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.805+0000 7f61be5fe700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b80af670 con 0x7f61b80cbc10 2023-07-06T01:19:27.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.805+0000 7f61be5fe700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b806a9e0 con 0x7f61b80af2a0 2023-07-06T01:19:27.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.805+0000 7f61be5fe700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b80cc7a0 con 0x7f61b806a000 2023-07-06T01:19:27.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.805+0000 7f61bcb9b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f61b80cbc10 0x7f61b80cc090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36012/0 (socket says 172.21.15.22:36012) 2023-07-06T01:19:27.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.805+0000 7f61bcb9b700 1 -- 172.21.15.22:0/676538146 learned_addr learned my addr 172.21.15.22:0/676538146 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:27.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.805+0000 7f61b7fff700 1 --2- 172.21.15.22:0/676538146 >> v2:172.21.15.103:3300/0 conn(0x7f61b80af2a0 0x7f61b806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:27.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.805+0000 7f61b77fe700 1 --2- 172.21.15.22:0/676538146 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b806a000 0x7f61b806a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:27.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.805+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717017516 0 0) 0x7f61b80af670 con 0x7f61b80cbc10 2023-07-06T01:19:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.805+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a40035b0 con 0x7f61b80cbc10 2023-07-06T01:19:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.805+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3696402036 0 0) 0x7f61a40035b0 con 0x7f61b80cbc10 2023-07-06T01:19:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b806a000 msgr2=0x7f61b806a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 --2- 172.21.15.22:0/676538146 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b806a000 0x7f61b806a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 >> v2:172.21.15.103:3300/0 conn(0x7f61b80af2a0 msgr2=0x7f61b806caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 --2- 172.21.15.22:0/676538146 >> v2:172.21.15.103:3300/0 conn(0x7f61b80af2a0 0x7f61b806caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b806d030 con 0x7f61b80cbc10 2023-07-06T01:19:27.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f61b00019f0 con 0x7f61b80cbc10 2023-07-06T01:19:27.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 >> v1:172.21.15.22:6789/0 conn(0x7f61b80cbc10 legacy=0x7f61b80cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:27.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 --2- 172.21.15.22:0/676538146 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b806a000 0x7f61a4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:27.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 --2- 172.21.15.22:0/676538146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61a4003f80 0x7f61a4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:27.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 --2- 172.21.15.22:0/676538146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61a4004d90 0x7f61a4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:27.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b80cc7a0 con 0x7f61a4004d90 2023-07-06T01:19:27.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a400ba10 con 0x7f61a4003f80 2023-07-06T01:19:27.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a400bb50 con 0x7f61b806a000 2023-07-06T01:19:27.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b7fff700 1 --2- 172.21.15.22:0/676538146 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b806a000 0x7f61a4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:27.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61bcb9b700 1 --2- 172.21.15.22:0/676538146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61a4004d90 0x7f61a4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:27.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b77fe700 1 --2- 172.21.15.22:0/676538146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61a4003f80 0x7f61a4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:27.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61bcb9b700 1 -- 172.21.15.22:0/676538146 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b806a000 msgr2=0x7f61a4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:27.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61bcb9b700 1 --2- 172.21.15.22:0/676538146 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b806a000 0x7f61a4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:27.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61bcb9b700 1 -- 172.21.15.22:0/676538146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61a4003f80 msgr2=0x7f61a4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:27.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61bcb9b700 1 --2- 172.21.15.22:0/676538146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61a4003f80 0x7f61a4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:27.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61bcb9b700 1 -- 172.21.15.22:0/676538146 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a40035b0 con 0x7f61a4004d90 2023-07-06T01:19:27.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b77fe700 1 --2- 172.21.15.22:0/676538146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61a4003f80 0x7f61a4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:27.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61bcb9b700 1 --2- 172.21.15.22:0/676538146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61a4004d90 0x7f61a4005200 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f61b000a860 tx=0x7f61a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebc2f04ce76e047 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:27.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0012600 con 0x7f61a4004d90 2023-07-06T01:19:27.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b0012780 con 0x7f61a4004d90 2023-07-06T01:19:27.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.806+0000 7f61b6ffd700 1 -- 172.21.15.22:0/676538146 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0012b30 con 0x7f61a4004d90 2023-07-06T01:19:27.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.807+0000 7f61be5fe700 1 -- 172.21.15.22:0/676538146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61a4004d90 msgr2=0x7f61a4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:27.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.807+0000 7f61be5fe700 1 --2- 172.21.15.22:0/676538146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61a4004d90 0x7f61a4005200 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f61b000a860 tx=0x7f61a4005880 comp rx=0 tx=0).stop 2023-07-06T01:19:27.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.807+0000 7f61bcb9b700 1 -- 172.21.15.22:0/676538146 reap_dead start 2023-07-06T01:19:27.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.807+0000 7f61be5fe700 1 -- 172.21.15.22:0/676538146 shutdown_connections 2023-07-06T01:19:27.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.807+0000 7f61be5fe700 1 -- 172.21.15.22:0/676538146 >> 172.21.15.22:0/676538146 conn(0x7f61b80b1890 msgr2=0x7f61b80b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:27.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.807+0000 7f61be5fe700 1 -- 172.21.15.22:0/676538146 shutdown_connections 2023-07-06T01:19:27.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.807+0000 7f61be5fe700 1 -- 172.21.15.22:0/676538146 wait complete. 2023-07-06T01:19:27.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.808+0000 7f61be5fe700 1 Processor -- start 2023-07-06T01:19:27.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.808+0000 7f61be5fe700 1 -- start start 2023-07-06T01:19:27.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.809+0000 7f61be5fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b81e6540 0x7f61b80b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:27.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.809+0000 7f61be5fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b81e6910 0x7f61b8159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:27.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.809+0000 7f61be5fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61b80b2c00 0x7f61b815a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:27.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.809+0000 7f61be5fe700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b81e8b80 con 0x7f61b80b2c00 2023-07-06T01:19:27.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.809+0000 7f61be5fe700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b80b3ef0 con 0x7f61b81e6540 2023-07-06T01:19:27.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.809+0000 7f61be5fe700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b80b41d0 con 0x7f61b81e6910 2023-07-06T01:19:27.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.809+0000 7f61bcb9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61b80b2c00 0x7f61b815a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:27.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.809+0000 7f61bcb9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61b80b2c00 0x7f61b815a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45280/0 (socket says 172.21.15.22:45280) 2023-07-06T01:19:27.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.809+0000 7f61bcb9b700 1 -- 172.21.15.22:0/2211811753 learned_addr learned my addr 172.21.15.22:0/2211811753 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:27.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.809+0000 7f61b77fe700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b81e6910 0x7f61b8159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:27.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.809+0000 7f61b7fff700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b81e6540 0x7f61b80b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:27.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.810+0000 7f61bcb9b700 1 -- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b81e6910 msgr2=0x7f61b8159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:27.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.810+0000 7f61bcb9b700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b81e6910 0x7f61b8159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:27.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.810+0000 7f61bcb9b700 1 -- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b81e6540 msgr2=0x7f61b80b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:27.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.810+0000 7f61bcb9b700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b81e6540 0x7f61b80b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:27.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.810+0000 7f61bcb9b700 1 -- 172.21.15.22:0/2211811753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b81e8e20 con 0x7f61b80b2c00 2023-07-06T01:19:27.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.810+0000 7f61b7fff700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b81e6540 0x7f61b80b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:27.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.810+0000 7f61b77fe700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b81e6910 0x7f61b8159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:27.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.810+0000 7f61bcb9b700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61b80b2c00 0x7f61b815a1d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f61b0001cb0 tx=0x7f61b0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.810+0000 7f61b4ff9700 1 -- 172.21.15.22:0/2211811753 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0026660 con 0x7f61b80b2c00 2023-07-06T01:19:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.810+0000 7f61b4ff9700 1 -- 172.21.15.22:0/2211811753 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b00267e0 con 0x7f61b80b2c00 2023-07-06T01:19:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.810+0000 7f61be5fe700 1 -- 172.21.15.22:0/2211811753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b815ccc0 con 0x7f61b80b2c00 2023-07-06T01:19:27.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.810+0000 7f61b4ff9700 1 -- 172.21.15.22:0/2211811753 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0026b90 con 0x7f61b80b2c00 2023-07-06T01:19:27.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.810+0000 7f61be5fe700 1 -- 172.21.15.22:0/2211811753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b80b44b0 con 0x7f61b80b2c00 2023-07-06T01:19:27.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.811+0000 7f61be5fe700 1 -- 172.21.15.22:0/2211811753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b80744b0 con 0x7f61b80b2c00 2023-07-06T01:19:27.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.811+0000 7f61b4ff9700 1 -- 172.21.15.22:0/2211811753 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f61b0019070 con 0x7f61b80b2c00 2023-07-06T01:19:27.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.812+0000 7f61b4ff9700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f619806ace0 0x7f619806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:27.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.812+0000 7f61b4ff9700 1 -- 172.21.15.22:0/2211811753 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61b008aab0 con 0x7f61b80b2c00 2023-07-06T01:19:27.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.812+0000 7f61b7fff700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f619806ace0 0x7f619806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:27.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.813+0000 7f61b7fff700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f619806ace0 0x7f619806d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009ce0 tx=0x7f61a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:27.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:27.816+0000 7f61b4ff9700 1 -- 172.21.15.22:0/2211811753 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f61b0055bf0 con 0x7f61b80b2c00 2023-07-06T01:19:28.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.058+0000 7f61be5fe700 1 -- 172.21.15.22:0/2211811753 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f61b80cfa80 con 0x7f61b80b2c00 2023-07-06T01:19:28.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.656+0000 7f61b4ff9700 1 -- 172.21.15.22:0/2211811753 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f61b00210c0 con 0x7f61b80b2c00 2023-07-06T01:19:28.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.659+0000 7f61be5fe700 1 -- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f619806ace0 msgr2=0x7f619806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:28.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.659+0000 7f61be5fe700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f619806ace0 0x7f619806d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009ce0 tx=0x7f61a8008040 comp rx=0 tx=0).stop 2023-07-06T01:19:28.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.659+0000 7f61be5fe700 1 -- 172.21.15.22:0/2211811753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61b80b2c00 msgr2=0x7f61b815a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:28.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.659+0000 7f61be5fe700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61b80b2c00 0x7f61b815a1d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f61b0001cb0 tx=0x7f61b0001d80 comp rx=0 tx=0).stop 2023-07-06T01:19:28.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.659+0000 7f61be5fe700 1 -- 172.21.15.22:0/2211811753 shutdown_connections 2023-07-06T01:19:28.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.659+0000 7f61be5fe700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61b80b2c00 0x7f61b815a1d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:28.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.659+0000 7f61be5fe700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f619806ace0 0x7f619806d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:28.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.659+0000 7f61be5fe700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61b81e6540 0x7f61b80b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:28.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.659+0000 7f61be5fe700 1 --2- 172.21.15.22:0/2211811753 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61b81e6910 0x7f61b8159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:28.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.659+0000 7f61be5fe700 1 -- 172.21.15.22:0/2211811753 >> 172.21.15.22:0/2211811753 conn(0x7f61b80b1890 msgr2=0x7f61b806e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:28.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.659+0000 7f61be5fe700 1 -- 172.21.15.22:0/2211811753 shutdown_connections 2023-07-06T01:19:28.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.659+0000 7f61be5fe700 1 -- 172.21.15.22:0/2211811753 wait complete. 2023-07-06T01:19:28.675 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T01:19:28.676 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:19:28.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.806+0000 7f1db7f25700 1 Processor -- start 2023-07-06T01:19:28.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.806+0000 7f1db7f25700 1 -- start start 2023-07-06T01:19:28.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.807+0000 7f1db7f25700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1db00c9a40 0x7f1db00c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:28.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.807+0000 7f1db7f25700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db00ca350 con 0x7f1db015a690 2023-07-06T01:19:28.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.807+0000 7f1db7f25700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db00cd640 con 0x7f1db006a000 2023-07-06T01:19:28.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.807+0000 7f1db7f25700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db00cd920 con 0x7f1db00c9a40 2023-07-06T01:19:28.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.807+0000 7f1db5cc1700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1db00c9a40 0x7f1db00c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:28.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.807+0000 7f1db54c0700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1db015a690 0x7f1db00ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36028/0 (socket says 172.21.15.22:36028) 2023-07-06T01:19:28.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.807+0000 7f1db54c0700 1 -- 172.21.15.22:0/1180101410 learned_addr learned my addr 172.21.15.22:0/1180101410 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:28.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.807+0000 7f1db5cc1700 1 -- 172.21.15.22:0/1180101410 >> v1:172.21.15.103:6789/0 conn(0x7f1db006a000 legacy=0x7f1db006a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.808+0000 7f1db5cc1700 1 -- 172.21.15.22:0/1180101410 >> v1:172.21.15.22:6789/0 conn(0x7f1db015a690 legacy=0x7f1db00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.808+0000 7f1db4cbf700 1 -- 172.21.15.22:0/1180101410 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100269051 0 0) 0x7f1db00ca350 con 0x7f1db015a690 2023-07-06T01:19:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.808+0000 7f1db5cc1700 1 -- 172.21.15.22:0/1180101410 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db00cdc00 con 0x7f1db00c9a40 2023-07-06T01:19:28.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.808+0000 7f1db5cc1700 1 --2- 172.21.15.22:0/1180101410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db00c9a40 0x7f1db00c9e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009840 tx=0x7f1da0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b47c0da3a6dfcb5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.808+0000 7f1db4cbf700 1 -- 172.21.15.22:0/1180101410 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0011830 con 0x7f1db00c9a40 2023-07-06T01:19:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.808+0000 7f1db4cbf700 1 -- 172.21.15.22:0/1180101410 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da0011990 con 0x7f1db00c9a40 2023-07-06T01:19:28.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.808+0000 7f1db4cbf700 1 -- 172.21.15.22:0/1180101410 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0011d20 con 0x7f1db00c9a40 2023-07-06T01:19:28.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.808+0000 7f1db7f25700 1 -- 172.21.15.22:0/1180101410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db00c9a40 msgr2=0x7f1db00c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:28.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.808+0000 7f1db7f25700 1 --2- 172.21.15.22:0/1180101410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db00c9a40 0x7f1db00c9e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009840 tx=0x7f1da0009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.808+0000 7f1db7f25700 1 -- 172.21.15.22:0/1180101410 shutdown_connections 2023-07-06T01:19:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.808+0000 7f1db7f25700 1 --2- 172.21.15.22:0/1180101410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db00c9a40 0x7f1db00c9e10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.808+0000 7f1db7f25700 1 -- 172.21.15.22:0/1180101410 >> 172.21.15.22:0/1180101410 conn(0x7f1db00ccb00 msgr2=0x7f1db00cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:28.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.809+0000 7f1db7f25700 1 -- 172.21.15.22:0/1180101410 shutdown_connections 2023-07-06T01:19:28.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.809+0000 7f1db7f25700 1 -- 172.21.15.22:0/1180101410 wait complete. 2023-07-06T01:19:28.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.810+0000 7f1db7f25700 1 Processor -- start 2023-07-06T01:19:28.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.810+0000 7f1db7f25700 1 -- start start 2023-07-06T01:19:28.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.810+0000 7f1db7f25700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db006a000 0x7f1db0154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:28.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.810+0000 7f1db7f25700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1db00c9a40 0x7f1db00b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:28.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.810+0000 7f1db7f25700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1db015a690 0x7f1db00b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:28.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.810+0000 7f1db7f25700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db0152490 con 0x7f1db006a000 2023-07-06T01:19:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.810+0000 7f1db7f25700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db0152fd0 con 0x7f1db015a690 2023-07-06T01:19:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.810+0000 7f1db7f25700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db01526a0 con 0x7f1db00c9a40 2023-07-06T01:19:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db5cc1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db006a000 0x7f1db0154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db5cc1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db006a000 0x7f1db0154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45290/0 (socket says 172.21.15.22:45290) 2023-07-06T01:19:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db5cc1700 1 -- 172.21.15.22:0/1271806550 learned_addr learned my addr 172.21.15.22:0/1271806550 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db54c0700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1db00c9a40 0x7f1db00b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db64c2700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1db015a690 0x7f1db00b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:28.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db5cc1700 1 -- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1db00c9a40 msgr2=0x7f1db00b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db5cc1700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1db00c9a40 0x7f1db00b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db5cc1700 1 -- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1db015a690 msgr2=0x7f1db00b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db5cc1700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1db015a690 0x7f1db00b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db5cc1700 1 -- 172.21.15.22:0/1271806550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da00094a0 con 0x7f1db006a000 2023-07-06T01:19:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db54c0700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1db00c9a40 0x7f1db00b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db5cc1700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db006a000 0x7f1db0154050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1da0014040 tx=0x7f1da0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:28.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1da67fc700 1 -- 172.21.15.22:0/1271806550 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da001a900 con 0x7f1db006a000 2023-07-06T01:19:28.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1da67fc700 1 -- 172.21.15.22:0/1271806550 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da001aa60 con 0x7f1db006a000 2023-07-06T01:19:28.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db7f25700 1 -- 172.21.15.22:0/1271806550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db0152980 con 0x7f1db006a000 2023-07-06T01:19:28.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1da67fc700 1 -- 172.21.15.22:0/1271806550 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da001add0 con 0x7f1db006a000 2023-07-06T01:19:28.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.811+0000 7f1db7f25700 1 -- 172.21.15.22:0/1271806550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db0153460 con 0x7f1db006a000 2023-07-06T01:19:28.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.812+0000 7f1db7f25700 1 -- 172.21.15.22:0/1271806550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db00744b0 con 0x7f1db006a000 2023-07-06T01:19:28.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.813+0000 7f1da67fc700 1 -- 172.21.15.22:0/1271806550 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1da0022070 con 0x7f1db006a000 2023-07-06T01:19:28.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.813+0000 7f1da67fc700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d9406ace0 0x7f1d9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:28.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.813+0000 7f1da67fc700 1 -- 172.21.15.22:0/1271806550 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1da008a740 con 0x7f1db006a000 2023-07-06T01:19:28.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.813+0000 7f1db54c0700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d9406ace0 0x7f1d9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:28.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.814+0000 7f1db54c0700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d9406ace0 0x7f1d9406d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1db013e590 tx=0x7f1dac007ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:28.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:28.817+0000 7f1da67fc700 1 -- 172.21.15.22:0/1271806550 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1da0055880 con 0x7f1db006a000 2023-07-06T01:19:29.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.058+0000 7f1db7f25700 1 -- 172.21.15.22:0/1271806550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1db00b5370 con 0x7f1db006a000 2023-07-06T01:19:29.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.058+0000 7f1da67fc700 1 -- 172.21.15.22:0/1271806550 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4671 (secure 0 0 0) 0x7f1da0006cc0 con 0x7f1db006a000 2023-07-06T01:19:29.061 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:29.061 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/2496681609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2496681609},{"type":"v1","addr":"172.21.15.22:6833","nonce":2496681609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2891594418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2891594418},{"type":"v1","addr":"172.21.15.22:6835","nonce":2891594418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/4097705589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4097705589},{"type":"v1","addr":"172.21.15.103:6836","nonce":4097705589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T01:19:25.042193+0000","modified":"2023-07-06T01:19:28.649724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6837/4015300665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":4015300665},{"type":"v1","addr":"172.21.15.103:6837","nonce":4015300665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T01:19:29.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.060+0000 7f1db7f25700 1 -- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d9406ace0 msgr2=0x7f1d9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.060+0000 7f1db7f25700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d9406ace0 0x7f1d9406d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1db013e590 tx=0x7f1dac007ce0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.060+0000 7f1db7f25700 1 -- 172.21.15.22:0/1271806550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db006a000 msgr2=0x7f1db0154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.060+0000 7f1db7f25700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db006a000 0x7f1db0154050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1da0014040 tx=0x7f1da0004750 comp rx=0 tx=0).stop 2023-07-06T01:19:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.060+0000 7f1db7f25700 1 -- 172.21.15.22:0/1271806550 shutdown_connections 2023-07-06T01:19:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.060+0000 7f1db7f25700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d9406ace0 0x7f1d9406d190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.060+0000 7f1db7f25700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1db015a690 0x7f1db00b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.060+0000 7f1db7f25700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1db006a000 0x7f1db0154050 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.060+0000 7f1db7f25700 1 --2- 172.21.15.22:0/1271806550 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1db00c9a40 0x7f1db00b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.060+0000 7f1db7f25700 1 -- 172.21.15.22:0/1271806550 >> 172.21.15.22:0/1271806550 conn(0x7f1db00ccb00 msgr2=0x7f1db00730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:29.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.061+0000 7f1db7f25700 1 -- 172.21.15.22:0/1271806550 shutdown_connections 2023-07-06T01:19:29.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.061+0000 7f1db7f25700 1 -- 172.21.15.22:0/1271806550 wait complete. 2023-07-06T01:19:29.064 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2023-07-06T01:19:29.075 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:29.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.199+0000 7fd98f701700 1 Processor -- start 2023-07-06T01:19:29.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.199+0000 7fd98f701700 1 -- start start 2023-07-06T01:19:29.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.200+0000 7fd98f701700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98806c900 0x7fd988163a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.200+0000 7fd98f701700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd988158f80 con 0x7fd988158bb0 2023-07-06T01:19:29.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.200+0000 7fd98f701700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98815ce90 con 0x7fd98806b5a0 2023-07-06T01:19:29.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.200+0000 7fd98f701700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880701e0 con 0x7fd98806c900 2023-07-06T01:19:29.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.200+0000 7fd98d49d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fd988158bb0 0x7fd98815c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36042/0 (socket says 172.21.15.22:36042) 2023-07-06T01:19:29.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.200+0000 7fd98d49d700 1 -- 172.21.15.22:0/3652724406 learned_addr learned my addr 172.21.15.22:0/3652724406 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:29.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.200+0000 7fd98dc9e700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98806c900 0x7fd988163a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.200+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096735579 0 0) 0x7fd988158f80 con 0x7fd988158bb0 2023-07-06T01:19:29.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.200+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9740035b0 con 0x7fd988158bb0 2023-07-06T01:19:29.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2150282268 0 0) 0x7fd9740035b0 con 0x7fd988158bb0 2023-07-06T01:19:29.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98806c900 msgr2=0x7fd988163a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:29.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98806c900 0x7fd988163a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 >> v1:172.21.15.103:6789/0 conn(0x7fd98806b5a0 legacy=0x7fd98806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98815d170 con 0x7fd988158bb0 2023-07-06T01:19:29.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7fd978001670 con 0x7fd988158bb0 2023-07-06T01:19:29.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 >> v1:172.21.15.22:6789/0 conn(0x7fd988158bb0 legacy=0x7fd98815c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd974003c00 0x7fd974003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98806c900 0x7fd97400a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd97400af90 0x7fd97400d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880701e0 con 0x7fd97400af90 2023-07-06T01:19:29.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97400dbf0 con 0x7fd974003c00 2023-07-06T01:19:29.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97400ddb0 con 0x7fd98806c900 2023-07-06T01:19:29.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd98dc9e700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd97400af90 0x7fd97400d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd98d49d700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd974003c00 0x7fd974003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd98cc9c700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98806c900 0x7fd97400a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd98dc9e700 1 -- 172.21.15.22:0/3652724406 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98806c900 msgr2=0x7fd97400a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:29.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd98dc9e700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98806c900 0x7fd97400a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd98dc9e700 1 -- 172.21.15.22:0/3652724406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd974003c00 msgr2=0x7fd974003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:29.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd98dc9e700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd974003c00 0x7fd974003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd98dc9e700 1 -- 172.21.15.22:0/3652724406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9740035b0 con 0x7fd97400af90 2023-07-06T01:19:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd98d49d700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd974003c00 0x7fd974003fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd98cc9c700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98806c900 0x7fd97400a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd98dc9e700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd97400af90 0x7fd97400d410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd980009f90 tx=0x7fd974003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d95f3a72909400c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98000ec30 con 0x7fd97400af90 2023-07-06T01:19:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98000f040 con 0x7fd97400af90 2023-07-06T01:19:29.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.201+0000 7fd97ffff700 1 -- 172.21.15.22:0/3652724406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980014480 con 0x7fd97400af90 2023-07-06T01:19:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.202+0000 7fd98f701700 1 -- 172.21.15.22:0/3652724406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd97400af90 msgr2=0x7fd97400d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.202+0000 7fd98f701700 1 --2- 172.21.15.22:0/3652724406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd97400af90 0x7fd97400d410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd980009f90 tx=0x7fd974003860 comp rx=0 tx=0).stop 2023-07-06T01:19:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.202+0000 7fd98dc9e700 1 -- 172.21.15.22:0/3652724406 reap_dead start 2023-07-06T01:19:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.202+0000 7fd98f701700 1 -- 172.21.15.22:0/3652724406 shutdown_connections 2023-07-06T01:19:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.202+0000 7fd98f701700 1 -- 172.21.15.22:0/3652724406 >> 172.21.15.22:0/3652724406 conn(0x7fd9880ccb00 msgr2=0x7fd9880cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:29.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.202+0000 7fd98f701700 1 -- 172.21.15.22:0/3652724406 shutdown_connections 2023-07-06T01:19:29.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.202+0000 7fd98f701700 1 -- 172.21.15.22:0/3652724406 wait complete. 2023-07-06T01:19:29.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.203+0000 7fd98f701700 1 Processor -- start 2023-07-06T01:19:29.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.203+0000 7fd98f701700 1 -- start start 2023-07-06T01:19:29.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98f701700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd97400af90 0x7fd9880b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98f701700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd98806a000 0x7fd98806a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98f701700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98814c1e0 0x7fd98814c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98f701700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98806a9b0 con 0x7fd97400af90 2023-07-06T01:19:29.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98f701700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880b5110 con 0x7fd98806a000 2023-07-06T01:19:29.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98f701700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880b3010 con 0x7fd98814c1e0 2023-07-06T01:19:29.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98d49d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd97400af90 0x7fd9880b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98d49d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd97400af90 0x7fd9880b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45310/0 (socket says 172.21.15.22:45310) 2023-07-06T01:19:29.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98d49d700 1 -- 172.21.15.22:0/850519729 learned_addr learned my addr 172.21.15.22:0/850519729 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:29.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98dc9e700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98814c1e0 0x7fd98814c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98cc9c700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd98806a000 0x7fd98806a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98d49d700 1 -- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98814c1e0 msgr2=0x7fd98814c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98d49d700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98814c1e0 0x7fd98814c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98d49d700 1 -- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd98806a000 msgr2=0x7fd98806a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98d49d700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd98806a000 0x7fd98806a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98d49d700 1 -- 172.21.15.22:0/850519729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9880b2a50 con 0x7fd97400af90 2023-07-06T01:19:29.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98cc9c700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd98806a000 0x7fd98806a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98dc9e700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98814c1e0 0x7fd98814c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.204+0000 7fd98d49d700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd97400af90 0x7fd9880b2220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd97800eb20 tx=0x7fd97800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.205+0000 7fd97dffb700 1 -- 172.21.15.22:0/850519729 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978005e00 con 0x7fd97400af90 2023-07-06T01:19:29.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.205+0000 7fd97dffb700 1 -- 172.21.15.22:0/850519729 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97800c780 con 0x7fd97400af90 2023-07-06T01:19:29.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.205+0000 7fd98f701700 1 -- 172.21.15.22:0/850519729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9880b2d30 con 0x7fd97400af90 2023-07-06T01:19:29.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.205+0000 7fd97dffb700 1 -- 172.21.15.22:0/850519729 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97800caf0 con 0x7fd97400af90 2023-07-06T01:19:29.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.205+0000 7fd98f701700 1 -- 172.21.15.22:0/850519729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd988156600 con 0x7fd97400af90 2023-07-06T01:19:29.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.206+0000 7fd98f701700 1 -- 172.21.15.22:0/850519729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9880744b0 con 0x7fd97400af90 2023-07-06T01:19:29.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.206+0000 7fd97dffb700 1 -- 172.21.15.22:0/850519729 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd97800cc50 con 0x7fd97400af90 2023-07-06T01:19:29.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.206+0000 7fd97dffb700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd96c06ace0 0x7fd96c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.206+0000 7fd97dffb700 1 -- 172.21.15.22:0/850519729 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd978089340 con 0x7fd97400af90 2023-07-06T01:19:29.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.207+0000 7fd98cc9c700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd96c06ace0 0x7fd96c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.207+0000 7fd98cc9c700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd96c06ace0 0x7fd96c06d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd984007fd0 tx=0x7fd984009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:29.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.211+0000 7fd97dffb700 1 -- 172.21.15.22:0/850519729 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd9780544b0 con 0x7fd97400af90 2023-07-06T01:19:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.409+0000 7fd98f701700 1 -- 172.21.15.22:0/850519729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9880b5370 con 0x7fd97400af90 2023-07-06T01:19:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.410+0000 7fd97dffb700 1 -- 172.21.15.22:0/850519729 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7fd978057ad0 con 0x7fd97400af90 2023-07-06T01:19:29.412 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:29.413 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:29.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.412+0000 7fd98f701700 1 -- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd96c06ace0 msgr2=0x7fd96c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.412+0000 7fd98f701700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd96c06ace0 0x7fd96c06d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd984007fd0 tx=0x7fd984009040 comp rx=0 tx=0).stop 2023-07-06T01:19:29.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.412+0000 7fd98f701700 1 -- 172.21.15.22:0/850519729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd97400af90 msgr2=0x7fd9880b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.412+0000 7fd98f701700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd97400af90 0x7fd9880b2220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd97800eb20 tx=0x7fd97800eee0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.412+0000 7fd98f701700 1 -- 172.21.15.22:0/850519729 shutdown_connections 2023-07-06T01:19:29.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.412+0000 7fd98f701700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd98814c1e0 0x7fd98814c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.412+0000 7fd98f701700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd97400af90 0x7fd9880b2220 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.412+0000 7fd98f701700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd96c06ace0 0x7fd96c06d190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.412+0000 7fd98f701700 1 --2- 172.21.15.22:0/850519729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd98806a000 0x7fd98806a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.412+0000 7fd98f701700 1 -- 172.21.15.22:0/850519729 >> 172.21.15.22:0/850519729 conn(0x7fd9880ccb00 msgr2=0x7fd9880cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:29.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.412+0000 7fd98f701700 1 -- 172.21.15.22:0/850519729 shutdown_connections 2023-07-06T01:19:29.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.413+0000 7fd98f701700 1 -- 172.21.15.22:0/850519729 wait complete. 2023-07-06T01:19:29.429 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T01:19:29.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:19:29.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.553+0000 7f45abd42700 1 Processor -- start 2023-07-06T01:19:29.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.553+0000 7f45abd42700 1 -- start start 2023-07-06T01:19:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45abd42700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f45a406a710 0x7f45a406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45abd42700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45a40cbc10 0x7f45a40cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45abd42700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a40af670 con 0x7f45a40af2a0 2023-07-06T01:19:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45abd42700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a40cd560 con 0x7f45a406a710 2023-07-06T01:19:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45abd42700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a415cab0 con 0x7f45a40cbc10 2023-07-06T01:19:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45a9ade700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f45a40af2a0 0x7f45a406a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36062/0 (socket says 172.21.15.22:36062) 2023-07-06T01:19:29.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45a9ade700 1 -- 172.21.15.22:0/2325447235 learned_addr learned my addr 172.21.15.22:0/2325447235 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45a92dd700 1 --2- 172.21.15.22:0/2325447235 >> v2:172.21.15.103:3300/0 conn(0x7f45a406a710 0x7f45a406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392270251 0 0) 0x7f45a40af670 con 0x7f45a40af2a0 2023-07-06T01:19:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45aa2df700 1 --2- 172.21.15.22:0/2325447235 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45a40cbc10 0x7f45a40cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45900035b0 con 0x7f45a40af2a0 2023-07-06T01:19:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 791591317 0 0) 0x7f45900035b0 con 0x7f45a40af2a0 2023-07-06T01:19:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45a40cbc10 msgr2=0x7f45a40cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45a8adc700 1 --2- 172.21.15.22:0/2325447235 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45a40cbc10 0x7f45a40cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 >> v2:172.21.15.103:3300/0 conn(0x7f45a406a710 msgr2=0x7f45a406caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45a8adc700 1 --2- 172.21.15.22:0/2325447235 >> v2:172.21.15.103:3300/0 conn(0x7f45a406a710 0x7f45a406caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a406d030 con 0x7f45a40af2a0 2023-07-06T01:19:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.554+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f45a0001670 con 0x7f45a40af2a0 2023-07-06T01:19:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 >> v1:172.21.15.22:6789/0 conn(0x7f45a40af2a0 legacy=0x7f45a406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a8adc700 1 --2- 172.21.15.22:0/2325447235 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45a40cbc10 0x7f4590003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a8adc700 1 --2- 172.21.15.22:0/2325447235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4590003f80 0x7f4590004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a8adc700 1 --2- 172.21.15.22:0/2325447235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4590004d90 0x7f4590005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40cd560 con 0x7f4590003f80 2023-07-06T01:19:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a415cab0 con 0x7f4590004d90 2023-07-06T01:19:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459000ba10 con 0x7f45a40cbc10 2023-07-06T01:19:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a92dd700 1 --2- 172.21.15.22:0/2325447235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4590003f80 0x7f4590004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a9ade700 1 --2- 172.21.15.22:0/2325447235 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45a40cbc10 0x7f4590003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45aa2df700 1 --2- 172.21.15.22:0/2325447235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4590004d90 0x7f4590005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a92dd700 1 -- 172.21.15.22:0/2325447235 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45a40cbc10 msgr2=0x7f4590003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:29.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a92dd700 1 --2- 172.21.15.22:0/2325447235 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45a40cbc10 0x7f4590003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a92dd700 1 -- 172.21.15.22:0/2325447235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4590004d90 msgr2=0x7f4590005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:29.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a92dd700 1 --2- 172.21.15.22:0/2325447235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4590004d90 0x7f4590005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a92dd700 1 -- 172.21.15.22:0/2325447235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45900035b0 con 0x7f4590003f80 2023-07-06T01:19:29.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a92dd700 1 --2- 172.21.15.22:0/2325447235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4590003f80 0x7f4590004850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4594009b30 tx=0x7f4590005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385924fae7b1dea7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:29.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459400e6c0 con 0x7f4590003f80 2023-07-06T01:19:29.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459400e820 con 0x7f4590003f80 2023-07-06T01:19:29.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.555+0000 7f45a8adc700 1 -- 172.21.15.22:0/2325447235 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459400eb90 con 0x7f4590003f80 2023-07-06T01:19:29.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.556+0000 7f45abd42700 1 -- 172.21.15.22:0/2325447235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4590003f80 msgr2=0x7f4590004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.556+0000 7f45abd42700 1 --2- 172.21.15.22:0/2325447235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4590003f80 0x7f4590004850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4594009b30 tx=0x7f4590005880 comp rx=0 tx=0).stop 2023-07-06T01:19:29.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.556+0000 7f45aa2df700 1 -- 172.21.15.22:0/2325447235 reap_dead start 2023-07-06T01:19:29.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.556+0000 7f45abd42700 1 -- 172.21.15.22:0/2325447235 shutdown_connections 2023-07-06T01:19:29.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.556+0000 7f45abd42700 1 -- 172.21.15.22:0/2325447235 >> 172.21.15.22:0/2325447235 conn(0x7f45a40b1890 msgr2=0x7f45a40b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:29.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.556+0000 7f45abd42700 1 -- 172.21.15.22:0/2325447235 shutdown_connections 2023-07-06T01:19:29.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.556+0000 7f45abd42700 1 -- 172.21.15.22:0/2325447235 wait complete. 2023-07-06T01:19:29.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.557+0000 7f45abd42700 1 Processor -- start 2023-07-06T01:19:29.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.557+0000 7f45abd42700 1 -- start start 2023-07-06T01:19:29.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45abd42700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a41f2a80 0x7f45a40c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45abd42700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45a41465a0 0x7f45a406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45abd42700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45a406e880 0x7f45a40c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45abd42700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a406ed20 con 0x7f45a41f2a80 2023-07-06T01:19:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45abd42700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40c5200 con 0x7f45a41465a0 2023-07-06T01:19:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45abd42700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40c54e0 con 0x7f45a406e880 2023-07-06T01:19:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45a9ade700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a41f2a80 0x7f45a40c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45a9ade700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a41f2a80 0x7f45a40c4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45330/0 (socket says 172.21.15.22:45330) 2023-07-06T01:19:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45a9ade700 1 -- 172.21.15.22:0/1441497632 learned_addr learned my addr 172.21.15.22:0/1441497632 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45a92dd700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45a41465a0 0x7f45a406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45aa2df700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45a406e880 0x7f45a40c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45a9ade700 1 -- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45a406e880 msgr2=0x7f45a40c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45a9ade700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45a406e880 0x7f45a40c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45a9ade700 1 -- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45a41465a0 msgr2=0x7f45a406e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45a9ade700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45a41465a0 0x7f45a406e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45a9ade700 1 -- 172.21.15.22:0/1441497632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a40c57c0 con 0x7f45a41f2a80 2023-07-06T01:19:29.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.558+0000 7f45a9ade700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a41f2a80 0x7f45a40c4b60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45a0007310 tx=0x7f45a0007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:29.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.559+0000 7f459a7fc700 1 -- 172.21.15.22:0/1441497632 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a000f5e0 con 0x7f45a41f2a80 2023-07-06T01:19:29.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.559+0000 7f459a7fc700 1 -- 172.21.15.22:0/1441497632 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a000f760 con 0x7f45a41f2a80 2023-07-06T01:19:29.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.559+0000 7f459a7fc700 1 -- 172.21.15.22:0/1441497632 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a000fb10 con 0x7f45a41f2a80 2023-07-06T01:19:29.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.559+0000 7f45abd42700 1 -- 172.21.15.22:0/1441497632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a40c5aa0 con 0x7f45a41f2a80 2023-07-06T01:19:29.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.559+0000 7f45abd42700 1 -- 172.21.15.22:0/1441497632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a40c62a0 con 0x7f45a41f2a80 2023-07-06T01:19:29.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.560+0000 7f45abd42700 1 -- 172.21.15.22:0/1441497632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a40c5fc0 con 0x7f45a41f2a80 2023-07-06T01:19:29.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.560+0000 7f459a7fc700 1 -- 172.21.15.22:0/1441497632 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f45a000fc70 con 0x7f45a41f2a80 2023-07-06T01:19:29.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.561+0000 7f459a7fc700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f458806ace0 0x7f458806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.561+0000 7f459a7fc700 1 -- 172.21.15.22:0/1441497632 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f45a00893a0 con 0x7f45a41f2a80 2023-07-06T01:19:29.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.561+0000 7f45a92dd700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f458806ace0 0x7f458806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.561+0000 7f45a92dd700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f458806ace0 0x7f458806d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45a4068cc0 tx=0x7f4594008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:29.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.565+0000 7f459a7fc700 1 -- 172.21.15.22:0/1441497632 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f45a40c5fc0 con 0x7f45a41f2a80 2023-07-06T01:19:29.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.758+0000 7f45abd42700 1 -- 172.21.15.22:0/1441497632 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45a40cccc0 con 0x7f458806ace0 2023-07-06T01:19:29.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.767+0000 7f459a7fc700 1 -- 172.21.15.22:0/1441497632 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282528 (secure 0 0 0) 0x7f459400ea10 con 0x7f458806ace0 2023-07-06T01:19:29.772 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:29.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.771+0000 7f45abd42700 1 -- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f458806ace0 msgr2=0x7f458806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.771+0000 7f45abd42700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f458806ace0 0x7f458806d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45a4068cc0 tx=0x7f4594008030 comp rx=0 tx=0).stop 2023-07-06T01:19:29.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.771+0000 7f45abd42700 1 -- 172.21.15.22:0/1441497632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a41f2a80 msgr2=0x7f45a40c4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.771+0000 7f45abd42700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a41f2a80 0x7f45a40c4b60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45a0007310 tx=0x7f45a0007620 comp rx=0 tx=0).stop 2023-07-06T01:19:29.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.771+0000 7f45abd42700 1 -- 172.21.15.22:0/1441497632 shutdown_connections 2023-07-06T01:19:29.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.771+0000 7f45abd42700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45a406e880 0x7f45a40c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.771+0000 7f45abd42700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a41f2a80 0x7f45a40c4b60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.771+0000 7f45abd42700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f458806ace0 0x7f458806d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.771+0000 7f45abd42700 1 --2- 172.21.15.22:0/1441497632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45a41465a0 0x7f45a406e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.771+0000 7f45abd42700 1 -- 172.21.15.22:0/1441497632 >> 172.21.15.22:0/1441497632 conn(0x7f45a40b1890 msgr2=0x7f45a415c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.771+0000 7f45abd42700 1 -- 172.21.15.22:0/1441497632 shutdown_connections 2023-07-06T01:19:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.771+0000 7f45abd42700 1 -- 172.21.15.22:0/1441497632 wait complete. 2023-07-06T01:19:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:19:29.785 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-07-06T01:19:27.998115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":185,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":150,"acting":150,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218960,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755760,"statfs":{"total":773094113280,"available":772869898240,"internally_reserved":0,"allocated":2785280,"data_stored":1743924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.376328"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068502+0000","last_change":"2023-07-06T01:19:25.653405+0000","last_active":"2023-07-06T01:19:27.068502+0000","last_peered":"2023-07-06T01:19:27.068502+0000","last_clean":"2023-07-06T01:19:27.068502+0000","last_became_active":"2023-07-06T01:19:25.653093+0000","last_became_peered":"2023-07-06T01:19:25.653093+0000","last_unstale":"2023-07-06T01:19:27.068502+0000","last_undegraded":"2023-07-06T01:19:27.068502+0000","last_fullsized":"2023-07-06T01:19:27.068502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:17.116328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632301+0000","last_change":"2023-07-06T01:19:25.649747+0000","last_active":"2023-07-06T01:19:26.632301+0000","last_peered":"2023-07-06T01:19:26.632301+0000","last_clean":"2023-07-06T01:19:26.632301+0000","last_became_active":"2023-07-06T01:19:25.649468+0000","last_became_peered":"2023-07-06T01:19:25.649468+0000","last_unstale":"2023-07-06T01:19:26.632301+0000","last_undegraded":"2023-07-06T01:19:26.632301+0000","last_fullsized":"2023-07-06T01:19:26.632301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:00.621635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068877+0000","last_change":"2023-07-06T01:19:25.647264+0000","last_active":"2023-07-06T01:19:27.068877+0000","last_peered":"2023-07-06T01:19:27.068877+0000","last_clean":"2023-07-06T01:19:27.068877+0000","last_became_active":"2023-07-06T01:19:25.646962+0000","last_became_peered":"2023-07-06T01:19:25.646962+0000","last_unstale":"2023-07-06T01:19:27.068877+0000","last_undegraded":"2023-07-06T01:19:27.068877+0000","last_fullsized":"2023-07-06T01:19:27.068877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:33.168360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631056+0000","last_change":"2023-07-06T01:19:25.655455+0000","last_active":"2023-07-06T01:19:26.631056+0000","last_peered":"2023-07-06T01:19:26.631056+0000","last_clean":"2023-07-06T01:19:26.631056+0000","last_became_active":"2023-07-06T01:19:25.655137+0000","last_became_peered":"2023-07-06T01:19:25.655137+0000","last_unstale":"2023-07-06T01:19:26.631056+0000","last_undegraded":"2023-07-06T01:19:26.631056+0000","last_fullsized":"2023-07-06T01:19:26.631056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:39.980297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631105+0000","last_change":"2023-07-06T01:19:25.657379+0000","last_active":"2023-07-06T01:19:26.631105+0000","last_peered":"2023-07-06T01:19:26.631105+0000","last_clean":"2023-07-06T01:19:26.631105+0000","last_became_active":"2023-07-06T01:19:25.657044+0000","last_became_peered":"2023-07-06T01:19:25.657044+0000","last_unstale":"2023-07-06T01:19:26.631105+0000","last_undegraded":"2023-07-06T01:19:26.631105+0000","last_fullsized":"2023-07-06T01:19:26.631105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:21:00.969228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630543+0000","last_change":"2023-07-06T01:19:25.654446+0000","last_active":"2023-07-06T01:19:26.630543+0000","last_peered":"2023-07-06T01:19:26.630543+0000","last_clean":"2023-07-06T01:19:26.630543+0000","last_became_active":"2023-07-06T01:19:25.653834+0000","last_became_peered":"2023-07-06T01:19:25.653834+0000","last_unstale":"2023-07-06T01:19:26.630543+0000","last_undegraded":"2023-07-06T01:19:26.630543+0000","last_fullsized":"2023-07-06T01:19:26.630543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:33.168360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632483+0000","last_change":"2023-07-06T01:19:25.648054+0000","last_active":"2023-07-06T01:19:26.632483+0000","last_peered":"2023-07-06T01:19:26.632483+0000","last_clean":"2023-07-06T01:19:26.632483+0000","last_became_active":"2023-07-06T01:19:25.647795+0000","last_became_peered":"2023-07-06T01:19:25.647795+0000","last_unstale":"2023-07-06T01:19:26.632483+0000","last_undegraded":"2023-07-06T01:19:26.632483+0000","last_fullsized":"2023-07-06T01:19:26.632483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068838+0000","last_change":"2023-07-06T01:19:25.660991+0000","last_active":"2023-07-06T01:19:27.068838+0000","last_peered":"2023-07-06T01:19:27.068838+0000","last_clean":"2023-07-06T01:19:27.068838+0000","last_became_active":"2023-07-06T01:19:25.660654+0000","last_became_peered":"2023-07-06T01:19:25.660654+0000","last_unstale":"2023-07-06T01:19:27.068838+0000","last_undegraded":"2023-07-06T01:19:27.068838+0000","last_fullsized":"2023-07-06T01:19:27.068838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:43:26.720400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068325+0000","last_change":"2023-07-06T01:19:25.654120+0000","last_active":"2023-07-06T01:19:27.068325+0000","last_peered":"2023-07-06T01:19:27.068325+0000","last_clean":"2023-07-06T01:19:27.068325+0000","last_became_active":"2023-07-06T01:19:25.653844+0000","last_became_peered":"2023-07-06T01:19:25.653844+0000","last_unstale":"2023-07-06T01:19:27.068325+0000","last_undegraded":"2023-07-06T01:19:27.068325+0000","last_fullsized":"2023-07-06T01:19:27.068325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:42:37.007207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631522+0000","last_change":"2023-07-06T01:19:25.650622+0000","last_active":"2023-07-06T01:19:26.631522+0000","last_peered":"2023-07-06T01:19:26.631522+0000","last_clean":"2023-07-06T01:19:26.631522+0000","last_became_active":"2023-07-06T01:19:25.650310+0000","last_became_peered":"2023-07-06T01:19:25.650310+0000","last_unstale":"2023-07-06T01:19:26.631522+0000","last_undegraded":"2023-07-06T01:19:26.631522+0000","last_fullsized":"2023-07-06T01:19:26.631522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:21.566043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630591+0000","last_change":"2023-07-06T01:19:25.654669+0000","last_active":"2023-07-06T01:19:26.630591+0000","last_peered":"2023-07-06T01:19:26.630591+0000","last_clean":"2023-07-06T01:19:26.630591+0000","last_became_active":"2023-07-06T01:19:25.654161+0000","last_became_peered":"2023-07-06T01:19:25.654161+0000","last_unstale":"2023-07-06T01:19:26.630591+0000","last_undegraded":"2023-07-06T01:19:26.630591+0000","last_fullsized":"2023-07-06T01:19:26.630591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:42.802038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068611+0000","last_change":"2023-07-06T01:19:25.645030+0000","last_active":"2023-07-06T01:19:27.068611+0000","last_peered":"2023-07-06T01:19:27.068611+0000","last_clean":"2023-07-06T01:19:27.068611+0000","last_became_active":"2023-07-06T01:19:25.644689+0000","last_became_peered":"2023-07-06T01:19:25.644689+0000","last_unstale":"2023-07-06T01:19:27.068611+0000","last_undegraded":"2023-07-06T01:19:27.068611+0000","last_fullsized":"2023-07-06T01:19:27.068611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:14.823559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630591+0000","last_change":"2023-07-06T01:19:25.636207+0000","last_active":"2023-07-06T01:19:26.630591+0000","last_peered":"2023-07-06T01:19:26.630591+0000","last_clean":"2023-07-06T01:19:26.630591+0000","last_became_active":"2023-07-06T01:19:25.635876+0000","last_became_peered":"2023-07-06T01:19:25.635876+0000","last_unstale":"2023-07-06T01:19:26.630591+0000","last_undegraded":"2023-07-06T01:19:26.630591+0000","last_fullsized":"2023-07-06T01:19:26.630591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:25.693365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944279+0000","last_change":"2023-07-06T01:19:25.655490+0000","last_active":"2023-07-06T01:19:26.944279+0000","last_peered":"2023-07-06T01:19:26.944279+0000","last_clean":"2023-07-06T01:19:26.944279+0000","last_became_active":"2023-07-06T01:19:25.655137+0000","last_became_peered":"2023-07-06T01:19:25.655137+0000","last_unstale":"2023-07-06T01:19:26.944279+0000","last_undegraded":"2023-07-06T01:19:26.944279+0000","last_fullsized":"2023-07-06T01:19:26.944279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:09.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068831+0000","last_change":"2023-07-06T01:19:25.644786+0000","last_active":"2023-07-06T01:19:27.068831+0000","last_peered":"2023-07-06T01:19:27.068831+0000","last_clean":"2023-07-06T01:19:27.068831+0000","last_became_active":"2023-07-06T01:19:25.644482+0000","last_became_peered":"2023-07-06T01:19:25.644482+0000","last_unstale":"2023-07-06T01:19:27.068831+0000","last_undegraded":"2023-07-06T01:19:27.068831+0000","last_fullsized":"2023-07-06T01:19:27.068831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:42.802038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068413+0000","last_change":"2023-07-06T01:19:25.642117+0000","last_active":"2023-07-06T01:19:27.068413+0000","last_peered":"2023-07-06T01:19:27.068413+0000","last_clean":"2023-07-06T01:19:27.068413+0000","last_became_active":"2023-07-06T01:19:25.641830+0000","last_became_peered":"2023-07-06T01:19:25.641830+0000","last_unstale":"2023-07-06T01:19:27.068413+0000","last_undegraded":"2023-07-06T01:19:27.068413+0000","last_fullsized":"2023-07-06T01:19:27.068413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:43:11.310891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631861+0000","last_change":"2023-07-06T01:19:25.656605+0000","last_active":"2023-07-06T01:19:26.631861+0000","last_peered":"2023-07-06T01:19:26.631861+0000","last_clean":"2023-07-06T01:19:26.631861+0000","last_became_active":"2023-07-06T01:19:25.656259+0000","last_became_peered":"2023-07-06T01:19:25.656259+0000","last_unstale":"2023-07-06T01:19:26.631861+0000","last_undegraded":"2023-07-06T01:19:26.631861+0000","last_fullsized":"2023-07-06T01:19:26.631861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:40.213207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:23.616180+0000","last_change":"2023-07-06T01:19:23.616180+0000","last_active":"2023-07-06T01:19:23.616180+0000","last_peered":"2023-07-06T01:19:23.616180+0000","last_clean":"2023-07-06T01:19:23.616180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:23.616180+0000","last_undegraded":"2023-07-06T01:19:23.616180+0000","last_fullsized":"2023-07-06T01:19:23.616180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944791+0000","last_change":"2023-07-06T01:19:25.650690+0000","last_active":"2023-07-06T01:19:26.944791+0000","last_peered":"2023-07-06T01:19:26.944791+0000","last_clean":"2023-07-06T01:19:26.944791+0000","last_became_active":"2023-07-06T01:19:25.650316+0000","last_became_peered":"2023-07-06T01:19:25.650316+0000","last_unstale":"2023-07-06T01:19:26.944791+0000","last_undegraded":"2023-07-06T01:19:26.944791+0000","last_fullsized":"2023-07-06T01:19:26.944791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:02.031653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068547+0000","last_change":"2023-07-06T01:19:24.645053+0000","last_active":"2023-07-06T01:19:27.068547+0000","last_peered":"2023-07-06T01:19:27.068547+0000","last_clean":"2023-07-06T01:19:27.068547+0000","last_became_active":"2023-07-06T01:19:24.644766+0000","last_became_peered":"2023-07-06T01:19:24.644766+0000","last_unstale":"2023-07-06T01:19:27.068547+0000","last_undegraded":"2023-07-06T01:19:27.068547+0000","last_fullsized":"2023-07-06T01:19:27.068547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:43.286430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630830+0000","last_change":"2023-07-06T01:19:25.649424+0000","last_active":"2023-07-06T01:19:26.630830+0000","last_peered":"2023-07-06T01:19:26.630830+0000","last_clean":"2023-07-06T01:19:26.630830+0000","last_became_active":"2023-07-06T01:19:25.649164+0000","last_became_peered":"2023-07-06T01:19:25.649164+0000","last_unstale":"2023-07-06T01:19:26.630830+0000","last_undegraded":"2023-07-06T01:19:26.630830+0000","last_fullsized":"2023-07-06T01:19:26.630830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:00.621635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944431+0000","last_change":"2023-07-06T01:19:24.645861+0000","last_active":"2023-07-06T01:19:26.944431+0000","last_peered":"2023-07-06T01:19:26.944431+0000","last_clean":"2023-07-06T01:19:26.944431+0000","last_became_active":"2023-07-06T01:19:24.645583+0000","last_became_peered":"2023-07-06T01:19:24.645583+0000","last_unstale":"2023-07-06T01:19:26.944431+0000","last_undegraded":"2023-07-06T01:19:26.944431+0000","last_fullsized":"2023-07-06T01:19:26.944431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:38.975082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069581+0000","last_change":"2023-07-06T01:19:25.656345+0000","last_active":"2023-07-06T01:19:27.069581+0000","last_peered":"2023-07-06T01:19:27.069581+0000","last_clean":"2023-07-06T01:19:27.069581+0000","last_became_active":"2023-07-06T01:19:25.655955+0000","last_became_peered":"2023-07-06T01:19:25.655955+0000","last_unstale":"2023-07-06T01:19:27.069581+0000","last_undegraded":"2023-07-06T01:19:27.069581+0000","last_fullsized":"2023-07-06T01:19:27.069581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:45.855998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630810+0000","last_change":"2023-07-06T01:19:24.643151+0000","last_active":"2023-07-06T01:19:26.630810+0000","last_peered":"2023-07-06T01:19:26.630810+0000","last_clean":"2023-07-06T01:19:26.630810+0000","last_became_active":"2023-07-06T01:19:24.642827+0000","last_became_peered":"2023-07-06T01:19:24.642827+0000","last_unstale":"2023-07-06T01:19:26.630810+0000","last_undegraded":"2023-07-06T01:19:26.630810+0000","last_fullsized":"2023-07-06T01:19:26.630810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:08.965590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944402+0000","last_change":"2023-07-06T01:19:25.652005+0000","last_active":"2023-07-06T01:19:26.944402+0000","last_peered":"2023-07-06T01:19:26.944402+0000","last_clean":"2023-07-06T01:19:26.944402+0000","last_became_active":"2023-07-06T01:19:25.651299+0000","last_became_peered":"2023-07-06T01:19:25.651299+0000","last_unstale":"2023-07-06T01:19:26.944402+0000","last_undegraded":"2023-07-06T01:19:26.944402+0000","last_fullsized":"2023-07-06T01:19:26.944402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632387+0000","last_change":"2023-07-06T01:19:24.642960+0000","last_active":"2023-07-06T01:19:26.632387+0000","last_peered":"2023-07-06T01:19:26.632387+0000","last_clean":"2023-07-06T01:19:26.632387+0000","last_became_active":"2023-07-06T01:19:24.642667+0000","last_became_peered":"2023-07-06T01:19:24.642667+0000","last_unstale":"2023-07-06T01:19:26.632387+0000","last_undegraded":"2023-07-06T01:19:26.632387+0000","last_fullsized":"2023-07-06T01:19:26.632387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:30.566175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944596+0000","last_change":"2023-07-06T01:19:25.650426+0000","last_active":"2023-07-06T01:19:26.944596+0000","last_peered":"2023-07-06T01:19:26.944596+0000","last_clean":"2023-07-06T01:19:26.944596+0000","last_became_active":"2023-07-06T01:19:25.650130+0000","last_became_peered":"2023-07-06T01:19:25.650130+0000","last_unstale":"2023-07-06T01:19:26.944596+0000","last_undegraded":"2023-07-06T01:19:26.944596+0000","last_fullsized":"2023-07-06T01:19:26.944596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:45:08.937322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:23.616180+0000","last_change":"2023-07-06T01:19:23.616180+0000","last_active":"2023-07-06T01:19:23.616180+0000","last_peered":"2023-07-06T01:19:23.616180+0000","last_clean":"2023-07-06T01:19:23.616180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:23.616180+0000","last_undegraded":"2023-07-06T01:19:23.616180+0000","last_fullsized":"2023-07-06T01:19:23.616180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068483+0000","last_change":"2023-07-06T01:19:25.652623+0000","last_active":"2023-07-06T01:19:27.068483+0000","last_peered":"2023-07-06T01:19:27.068483+0000","last_clean":"2023-07-06T01:19:27.068483+0000","last_became_active":"2023-07-06T01:19:25.652341+0000","last_became_peered":"2023-07-06T01:19:25.652341+0000","last_unstale":"2023-07-06T01:19:27.068483+0000","last_undegraded":"2023-07-06T01:19:27.068483+0000","last_fullsized":"2023-07-06T01:19:27.068483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:21:00.969228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:23.616180+0000","last_change":"2023-07-06T01:19:23.616180+0000","last_active":"2023-07-06T01:19:23.616180+0000","last_peered":"2023-07-06T01:19:23.616180+0000","last_clean":"2023-07-06T01:19:23.616180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:23.616180+0000","last_undegraded":"2023-07-06T01:19:23.616180+0000","last_fullsized":"2023-07-06T01:19:23.616180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:23.616180+0000","last_change":"2023-07-06T01:19:23.616180+0000","last_active":"2023-07-06T01:19:23.616180+0000","last_peered":"2023-07-06T01:19:23.616180+0000","last_clean":"2023-07-06T01:19:23.616180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:23.616180+0000","last_undegraded":"2023-07-06T01:19:23.616180+0000","last_fullsized":"2023-07-06T01:19:23.616180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944680+0000","last_change":"2023-07-06T01:19:25.651034+0000","last_active":"2023-07-06T01:19:26.944680+0000","last_peered":"2023-07-06T01:19:26.944680+0000","last_clean":"2023-07-06T01:19:26.944680+0000","last_became_active":"2023-07-06T01:19:25.650753+0000","last_became_peered":"2023-07-06T01:19:25.650753+0000","last_unstale":"2023-07-06T01:19:26.944680+0000","last_undegraded":"2023-07-06T01:19:26.944680+0000","last_fullsized":"2023-07-06T01:19:26.944680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:36.632721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630793+0000","last_change":"2023-07-06T01:19:24.648974+0000","last_active":"2023-07-06T01:19:26.630793+0000","last_peered":"2023-07-06T01:19:26.630793+0000","last_clean":"2023-07-06T01:19:26.630793+0000","last_became_active":"2023-07-06T01:19:24.648691+0000","last_became_peered":"2023-07-06T01:19:24.648691+0000","last_unstale":"2023-07-06T01:19:26.630793+0000","last_undegraded":"2023-07-06T01:19:26.630793+0000","last_fullsized":"2023-07-06T01:19:26.630793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:44.850783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631977+0000","last_change":"2023-07-06T01:19:25.656530+0000","last_active":"2023-07-06T01:19:26.631977+0000","last_peered":"2023-07-06T01:19:26.631977+0000","last_clean":"2023-07-06T01:19:26.631977+0000","last_became_active":"2023-07-06T01:19:25.655966+0000","last_became_peered":"2023-07-06T01:19:25.655966+0000","last_unstale":"2023-07-06T01:19:26.631977+0000","last_undegraded":"2023-07-06T01:19:26.631977+0000","last_fullsized":"2023-07-06T01:19:26.631977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:10.760963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:23.616180+0000","last_change":"2023-07-06T01:19:23.616180+0000","last_active":"2023-07-06T01:19:23.616180+0000","last_peered":"2023-07-06T01:19:23.616180+0000","last_clean":"2023-07-06T01:19:23.616180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:23.616180+0000","last_undegraded":"2023-07-06T01:19:23.616180+0000","last_fullsized":"2023-07-06T01:19:23.616180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630689+0000","last_change":"2023-07-06T01:19:25.646210+0000","last_active":"2023-07-06T01:19:26.630689+0000","last_peered":"2023-07-06T01:19:26.630689+0000","last_clean":"2023-07-06T01:19:26.630689+0000","last_became_active":"2023-07-06T01:19:25.645912+0000","last_became_peered":"2023-07-06T01:19:25.645912+0000","last_unstale":"2023-07-06T01:19:26.630689+0000","last_undegraded":"2023-07-06T01:19:26.630689+0000","last_fullsized":"2023-07-06T01:19:26.630689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:21.566043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068646+0000","last_change":"2023-07-06T01:19:24.642544+0000","last_active":"2023-07-06T01:19:27.068646+0000","last_peered":"2023-07-06T01:19:27.068646+0000","last_clean":"2023-07-06T01:19:27.068646+0000","last_became_active":"2023-07-06T01:19:24.642267+0000","last_became_peered":"2023-07-06T01:19:24.642267+0000","last_unstale":"2023-07-06T01:19:27.068646+0000","last_undegraded":"2023-07-06T01:19:27.068646+0000","last_fullsized":"2023-07-06T01:19:27.068646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:20:59.964013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944798+0000","last_change":"2023-07-06T01:19:25.650388+0000","last_active":"2023-07-06T01:19:26.944798+0000","last_peered":"2023-07-06T01:19:26.944798+0000","last_clean":"2023-07-06T01:19:26.944798+0000","last_became_active":"2023-07-06T01:19:25.650047+0000","last_became_peered":"2023-07-06T01:19:25.650047+0000","last_unstale":"2023-07-06T01:19:26.944798+0000","last_undegraded":"2023-07-06T01:19:26.944798+0000","last_fullsized":"2023-07-06T01:19:26.944798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:08:02.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:23.616180+0000","last_change":"2023-07-06T01:19:23.616180+0000","last_active":"2023-07-06T01:19:23.616180+0000","last_peered":"2023-07-06T01:19:23.616180+0000","last_clean":"2023-07-06T01:19:23.616180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:23.616180+0000","last_undegraded":"2023-07-06T01:19:23.616180+0000","last_fullsized":"2023-07-06T01:19:23.616180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631268+0000","last_change":"2023-07-06T01:19:25.649637+0000","last_active":"2023-07-06T01:19:26.631268+0000","last_peered":"2023-07-06T01:19:26.631268+0000","last_clean":"2023-07-06T01:19:26.631268+0000","last_became_active":"2023-07-06T01:19:25.649363+0000","last_became_peered":"2023-07-06T01:19:25.649363+0000","last_unstale":"2023-07-06T01:19:26.631268+0000","last_undegraded":"2023-07-06T01:19:26.631268+0000","last_fullsized":"2023-07-06T01:19:26.631268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:44.291645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:23.616180+0000","last_change":"2023-07-06T01:19:23.616180+0000","last_active":"2023-07-06T01:19:23.616180+0000","last_peered":"2023-07-06T01:19:23.616180+0000","last_clean":"2023-07-06T01:19:23.616180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:23.616180+0000","last_undegraded":"2023-07-06T01:19:23.616180+0000","last_fullsized":"2023-07-06T01:19:23.616180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:23.616180+0000","last_change":"2023-07-06T01:19:23.616180+0000","last_active":"2023-07-06T01:19:23.616180+0000","last_peered":"2023-07-06T01:19:23.616180+0000","last_clean":"2023-07-06T01:19:23.616180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:23.616180+0000","last_undegraded":"2023-07-06T01:19:23.616180+0000","last_fullsized":"2023-07-06T01:19:23.616180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069126+0000","last_change":"2023-07-06T01:19:25.652728+0000","last_active":"2023-07-06T01:19:27.069126+0000","last_peered":"2023-07-06T01:19:27.069126+0000","last_clean":"2023-07-06T01:19:27.069126+0000","last_became_active":"2023-07-06T01:19:25.652420+0000","last_became_peered":"2023-07-06T01:19:25.652420+0000","last_unstale":"2023-07-06T01:19:27.069126+0000","last_undegraded":"2023-07-06T01:19:27.069126+0000","last_fullsized":"2023-07-06T01:19:27.069126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:40.213207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631482+0000","last_change":"2023-07-06T01:19:24.640722+0000","last_active":"2023-07-06T01:19:26.631482+0000","last_peered":"2023-07-06T01:19:26.631482+0000","last_clean":"2023-07-06T01:19:26.631482+0000","last_became_active":"2023-07-06T01:19:24.640417+0000","last_became_peered":"2023-07-06T01:19:24.640417+0000","last_unstale":"2023-07-06T01:19:26.631482+0000","last_undegraded":"2023-07-06T01:19:26.631482+0000","last_fullsized":"2023-07-06T01:19:26.631482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:24.688149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630955+0000","last_change":"2023-07-06T01:19:25.636624+0000","last_active":"2023-07-06T01:19:26.630955+0000","last_peered":"2023-07-06T01:19:26.630955+0000","last_clean":"2023-07-06T01:19:26.630955+0000","last_became_active":"2023-07-06T01:19:25.636320+0000","last_became_peered":"2023-07-06T01:19:25.636320+0000","last_unstale":"2023-07-06T01:19:26.630955+0000","last_undegraded":"2023-07-06T01:19:26.630955+0000","last_fullsized":"2023-07-06T01:19:26.630955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944677+0000","last_change":"2023-07-06T01:19:24.645728+0000","last_active":"2023-07-06T01:19:26.944677+0000","last_peered":"2023-07-06T01:19:26.944677+0000","last_clean":"2023-07-06T01:19:26.944677+0000","last_became_active":"2023-07-06T01:19:24.644479+0000","last_became_peered":"2023-07-06T01:19:24.644479+0000","last_unstale":"2023-07-06T01:19:26.944677+0000","last_undegraded":"2023-07-06T01:19:26.944677+0000","last_fullsized":"2023-07-06T01:19:26.944677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:58.274250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069265+0000","last_change":"2023-07-06T01:19:24.649526+0000","last_active":"2023-07-06T01:19:27.069265+0000","last_peered":"2023-07-06T01:19:27.069265+0000","last_clean":"2023-07-06T01:19:27.069265+0000","last_became_active":"2023-07-06T01:19:24.648970+0000","last_became_peered":"2023-07-06T01:19:24.648970+0000","last_unstale":"2023-07-06T01:19:27.069265+0000","last_undegraded":"2023-07-06T01:19:27.069265+0000","last_fullsized":"2023-07-06T01:19:27.069265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:41.796823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:23.616180+0000","last_change":"2023-07-06T01:19:23.616180+0000","last_active":"2023-07-06T01:19:23.616180+0000","last_peered":"2023-07-06T01:19:23.616180+0000","last_clean":"2023-07-06T01:19:23.616180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:23.616180+0000","last_undegraded":"2023-07-06T01:19:23.616180+0000","last_fullsized":"2023-07-06T01:19:23.616180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944291+0000","last_change":"2023-07-06T01:19:24.645128+0000","last_active":"2023-07-06T01:19:26.944291+0000","last_peered":"2023-07-06T01:19:26.944291+0000","last_clean":"2023-07-06T01:19:26.944291+0000","last_became_active":"2023-07-06T01:19:24.644825+0000","last_became_peered":"2023-07-06T01:19:24.644825+0000","last_unstale":"2023-07-06T01:19:26.944291+0000","last_undegraded":"2023-07-06T01:19:26.944291+0000","last_fullsized":"2023-07-06T01:19:26.944291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:14:59.616420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069000+0000","last_change":"2023-07-06T01:19:25.654566+0000","last_active":"2023-07-06T01:19:27.069000+0000","last_peered":"2023-07-06T01:19:27.069000+0000","last_clean":"2023-07-06T01:19:27.069000+0000","last_became_active":"2023-07-06T01:19:25.654238+0000","last_became_peered":"2023-07-06T01:19:25.654238+0000","last_unstale":"2023-07-06T01:19:27.069000+0000","last_undegraded":"2023-07-06T01:19:27.069000+0000","last_fullsized":"2023-07-06T01:19:27.069000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:45.855998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944614+0000","last_change":"2023-07-06T01:19:24.648921+0000","last_active":"2023-07-06T01:19:26.944614+0000","last_peered":"2023-07-06T01:19:26.944614+0000","last_clean":"2023-07-06T01:19:26.944614+0000","last_became_active":"2023-07-06T01:19:24.648676+0000","last_became_peered":"2023-07-06T01:19:24.648676+0000","last_unstale":"2023-07-06T01:19:26.944614+0000","last_undegraded":"2023-07-06T01:19:26.944614+0000","last_fullsized":"2023-07-06T01:19:26.944614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:41:33.528500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-07-06T01:19:20.607493+0000","last_change":"2023-07-06T01:19:20.074195+0000","last_active":"2023-07-06T01:19:20.607493+0000","last_peered":"2023-07-06T01:19:20.607493+0000","last_clean":"2023-07-06T01:19:20.607493+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:19:20.607493+0000","last_undegraded":"2023-07-06T01:19:20.607493+0000","last_fullsized":"2023-07-06T01:19:20.607493+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:26:40.424511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944214+0000","last_change":"2023-07-06T01:19:25.652132+0000","last_active":"2023-07-06T01:19:26.944214+0000","last_peered":"2023-07-06T01:19:26.944214+0000","last_clean":"2023-07-06T01:19:26.944214+0000","last_became_active":"2023-07-06T01:19:25.651678+0000","last_became_peered":"2023-07-06T01:19:25.651678+0000","last_unstale":"2023-07-06T01:19:26.944214+0000","last_undegraded":"2023-07-06T01:19:26.944214+0000","last_fullsized":"2023-07-06T01:19:26.944214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:45.855998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944468+0000","last_change":"2023-07-06T01:19:19.587110+0000","last_active":"2023-07-06T01:19:26.944468+0000","last_peered":"2023-07-06T01:19:26.944468+0000","last_clean":"2023-07-06T01:19:26.944468+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:19:26.944468+0000","last_undegraded":"2023-07-06T01:19:26.944468+0000","last_fullsized":"2023-07-06T01:19:26.944468+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:12.165222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630881+0000","last_change":"2023-07-06T01:19:24.640259+0000","last_active":"2023-07-06T01:19:26.630881+0000","last_peered":"2023-07-06T01:19:26.630881+0000","last_clean":"2023-07-06T01:19:26.630881+0000","last_became_active":"2023-07-06T01:19:24.639942+0000","last_became_peered":"2023-07-06T01:19:24.639942+0000","last_unstale":"2023-07-06T01:19:26.630881+0000","last_undegraded":"2023-07-06T01:19:26.630881+0000","last_fullsized":"2023-07-06T01:19:26.630881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:30.566175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068648+0000","last_change":"2023-07-06T01:19:19.584242+0000","last_active":"2023-07-06T01:19:27.068648+0000","last_peered":"2023-07-06T01:19:27.068648+0000","last_clean":"2023-07-06T01:19:27.068648+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:19:27.068648+0000","last_undegraded":"2023-07-06T01:19:27.068648+0000","last_fullsized":"2023-07-06T01:19:27.068648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:27.113987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-07-06T01:19:20.699562+0000","last_change":"2023-07-06T01:19:19.586495+0000","last_active":"2023-07-06T01:19:20.699562+0000","last_peered":"2023-07-06T01:19:20.699562+0000","last_clean":"2023-07-06T01:19:20.699562+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:19:20.699562+0000","last_undegraded":"2023-07-06T01:19:20.699562+0000","last_fullsized":"2023-07-06T01:19:20.699562+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:54:15.978191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630456+0000","last_change":"2023-07-06T01:19:24.642436+0000","last_active":"2023-07-06T01:19:26.630456+0000","last_peered":"2023-07-06T01:19:26.630456+0000","last_clean":"2023-07-06T01:19:26.630456+0000","last_became_active":"2023-07-06T01:19:24.641979+0000","last_became_peered":"2023-07-06T01:19:24.641979+0000","last_unstale":"2023-07-06T01:19:26.630456+0000","last_undegraded":"2023-07-06T01:19:26.630456+0000","last_fullsized":"2023-07-06T01:19:26.630456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:43.286430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630728+0000","last_change":"2023-07-06T01:19:25.639313+0000","last_active":"2023-07-06T01:19:26.630728+0000","last_peered":"2023-07-06T01:19:26.630728+0000","last_clean":"2023-07-06T01:19:26.630728+0000","last_became_active":"2023-07-06T01:19:25.639042+0000","last_became_peered":"2023-07-06T01:19:25.639042+0000","last_unstale":"2023-07-06T01:19:26.630728+0000","last_undegraded":"2023-07-06T01:19:26.630728+0000","last_fullsized":"2023-07-06T01:19:26.630728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:40.213207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-07-06T01:19:20.607486+0000","last_change":"2023-07-06T01:19:19.586426+0000","last_active":"2023-07-06T01:19:20.607486+0000","last_peered":"2023-07-06T01:19:20.607486+0000","last_clean":"2023-07-06T01:19:20.607486+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:19:20.607486+0000","last_undegraded":"2023-07-06T01:19:20.607486+0000","last_fullsized":"2023-07-06T01:19:20.607486+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:38:06.964704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068473+0000","last_change":"2023-07-06T01:19:24.643437+0000","last_active":"2023-07-06T01:19:27.068473+0000","last_peered":"2023-07-06T01:19:27.068473+0000","last_clean":"2023-07-06T01:19:27.068473+0000","last_became_active":"2023-07-06T01:19:24.642926+0000","last_became_peered":"2023-07-06T01:19:24.642926+0000","last_unstale":"2023-07-06T01:19:27.068473+0000","last_undegraded":"2023-07-06T01:19:27.068473+0000","last_fullsized":"2023-07-06T01:19:27.068473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:17.543019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068999+0000","last_change":"2023-07-06T01:19:25.656081+0000","last_active":"2023-07-06T01:19:27.068999+0000","last_peered":"2023-07-06T01:19:27.068999+0000","last_clean":"2023-07-06T01:19:27.068999+0000","last_became_active":"2023-07-06T01:19:25.655801+0000","last_became_peered":"2023-07-06T01:19:25.655801+0000","last_unstale":"2023-07-06T01:19:27.068999+0000","last_undegraded":"2023-07-06T01:19:27.068999+0000","last_fullsized":"2023-07-06T01:19:27.068999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:10.760963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-07-06T01:19:20.607585+0000","last_change":"2023-07-06T01:19:19.586626+0000","last_active":"2023-07-06T01:19:20.607585+0000","last_peered":"2023-07-06T01:19:20.607585+0000","last_clean":"2023-07-06T01:19:20.607585+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:19:20.607585+0000","last_undegraded":"2023-07-06T01:19:20.607585+0000","last_fullsized":"2023-07-06T01:19:20.607585+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:08:03.634620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944310+0000","last_change":"2023-07-06T01:19:24.644389+0000","last_active":"2023-07-06T01:19:26.944310+0000","last_peered":"2023-07-06T01:19:26.944310+0000","last_clean":"2023-07-06T01:19:26.944310+0000","last_became_active":"2023-07-06T01:19:24.644113+0000","last_became_peered":"2023-07-06T01:19:24.644113+0000","last_unstale":"2023-07-06T01:19:26.944310+0000","last_undegraded":"2023-07-06T01:19:26.944310+0000","last_fullsized":"2023-07-06T01:19:26.944310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:30.566175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069023+0000","last_change":"2023-07-06T01:19:25.654127+0000","last_active":"2023-07-06T01:19:27.069023+0000","last_peered":"2023-07-06T01:19:27.069023+0000","last_clean":"2023-07-06T01:19:27.069023+0000","last_became_active":"2023-07-06T01:19:25.653903+0000","last_became_peered":"2023-07-06T01:19:25.653903+0000","last_unstale":"2023-07-06T01:19:27.069023+0000","last_undegraded":"2023-07-06T01:19:27.069023+0000","last_fullsized":"2023-07-06T01:19:27.069023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:25.693365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631434+0000","last_change":"2023-07-06T01:19:19.584086+0000","last_active":"2023-07-06T01:19:26.631434+0000","last_peered":"2023-07-06T01:19:26.631434+0000","last_clean":"2023-07-06T01:19:26.631434+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:19:26.631434+0000","last_undegraded":"2023-07-06T01:19:26.631434+0000","last_fullsized":"2023-07-06T01:19:26.631434+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:36.747665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:23.616180+0000","last_change":"2023-07-06T01:19:23.616180+0000","last_active":"2023-07-06T01:19:23.616180+0000","last_peered":"2023-07-06T01:19:23.616180+0000","last_clean":"2023-07-06T01:19:23.616180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:23.616180+0000","last_undegraded":"2023-07-06T01:19:23.616180+0000","last_fullsized":"2023-07-06T01:19:23.616180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069089+0000","last_change":"2023-07-06T01:19:25.641885+0000","last_active":"2023-07-06T01:19:27.069089+0000","last_peered":"2023-07-06T01:19:27.069089+0000","last_clean":"2023-07-06T01:19:27.069089+0000","last_became_active":"2023-07-06T01:19:25.641570+0000","last_became_peered":"2023-07-06T01:19:25.641570+0000","last_unstale":"2023-07-06T01:19:27.069089+0000","last_undegraded":"2023-07-06T01:19:27.069089+0000","last_fullsized":"2023-07-06T01:19:27.069089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:09.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069072+0000","last_change":"2023-07-06T01:19:19.586424+0000","last_active":"2023-07-06T01:19:27.069072+0000","last_peered":"2023-07-06T01:19:27.069072+0000","last_clean":"2023-07-06T01:19:27.069072+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:19:27.069072+0000","last_undegraded":"2023-07-06T01:19:27.069072+0000","last_fullsized":"2023-07-06T01:19:27.069072+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:15.511670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068948+0000","last_change":"2023-07-06T01:19:24.644635+0000","last_active":"2023-07-06T01:19:27.068948+0000","last_peered":"2023-07-06T01:19:27.068948+0000","last_clean":"2023-07-06T01:19:27.068948+0000","last_became_active":"2023-07-06T01:19:24.644344+0000","last_became_peered":"2023-07-06T01:19:24.644344+0000","last_unstale":"2023-07-06T01:19:27.068948+0000","last_undegraded":"2023-07-06T01:19:27.068948+0000","last_fullsized":"2023-07-06T01:19:27.068948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:39.207991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631123+0000","last_change":"2023-07-06T01:19:25.657020+0000","last_active":"2023-07-06T01:19:26.631123+0000","last_peered":"2023-07-06T01:19:26.631123+0000","last_clean":"2023-07-06T01:19:26.631123+0000","last_became_active":"2023-07-06T01:19:25.656084+0000","last_became_peered":"2023-07-06T01:19:25.656084+0000","last_unstale":"2023-07-06T01:19:26.631123+0000","last_undegraded":"2023-07-06T01:19:26.631123+0000","last_fullsized":"2023-07-06T01:19:26.631123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:22.147924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069477+0000","last_change":"2023-07-06T01:19:19.583699+0000","last_active":"2023-07-06T01:19:27.069477+0000","last_peered":"2023-07-06T01:19:27.069477+0000","last_clean":"2023-07-06T01:19:27.069477+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:19:27.069477+0000","last_undegraded":"2023-07-06T01:19:27.069477+0000","last_fullsized":"2023-07-06T01:19:27.069477+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:04.706590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631992+0000","last_change":"2023-07-06T01:19:24.642998+0000","last_active":"2023-07-06T01:19:26.631992+0000","last_peered":"2023-07-06T01:19:26.631992+0000","last_clean":"2023-07-06T01:19:26.631992+0000","last_became_active":"2023-07-06T01:19:24.642674+0000","last_became_peered":"2023-07-06T01:19:24.642674+0000","last_unstale":"2023-07-06T01:19:26.631992+0000","last_undegraded":"2023-07-06T01:19:26.631992+0000","last_fullsized":"2023-07-06T01:19:26.631992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:44.850783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068822+0000","last_change":"2023-07-06T01:19:25.651577+0000","last_active":"2023-07-06T01:19:27.068822+0000","last_peered":"2023-07-06T01:19:27.068822+0000","last_clean":"2023-07-06T01:19:27.068822+0000","last_became_active":"2023-07-06T01:19:25.651299+0000","last_became_peered":"2023-07-06T01:19:25.651299+0000","last_unstale":"2023-07-06T01:19:27.068822+0000","last_undegraded":"2023-07-06T01:19:27.068822+0000","last_fullsized":"2023-07-06T01:19:27.068822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:25.693365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631194+0000","last_change":"2023-07-06T01:19:24.648687+0000","last_active":"2023-07-06T01:19:26.631194+0000","last_peered":"2023-07-06T01:19:26.631194+0000","last_clean":"2023-07-06T01:19:26.631194+0000","last_became_active":"2023-07-06T01:19:24.648363+0000","last_became_peered":"2023-07-06T01:19:24.648363+0000","last_unstale":"2023-07-06T01:19:26.631194+0000","last_undegraded":"2023-07-06T01:19:26.631194+0000","last_fullsized":"2023-07-06T01:19:26.631194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:24.719467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069057+0000","last_change":"2023-07-06T01:19:25.646102+0000","last_active":"2023-07-06T01:19:27.069057+0000","last_peered":"2023-07-06T01:19:27.069057+0000","last_clean":"2023-07-06T01:19:27.069057+0000","last_became_active":"2023-07-06T01:19:25.645911+0000","last_became_peered":"2023-07-06T01:19:25.645911+0000","last_unstale":"2023-07-06T01:19:27.069057+0000","last_undegraded":"2023-07-06T01:19:27.069057+0000","last_fullsized":"2023-07-06T01:19:27.069057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:58:05.681285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631273+0000","last_change":"2023-07-06T01:19:24.645915+0000","last_active":"2023-07-06T01:19:26.631273+0000","last_peered":"2023-07-06T01:19:26.631273+0000","last_clean":"2023-07-06T01:19:26.631273+0000","last_became_active":"2023-07-06T01:19:24.645626+0000","last_became_peered":"2023-07-06T01:19:24.645626+0000","last_unstale":"2023-07-06T01:19:26.631273+0000","last_undegraded":"2023-07-06T01:19:26.631273+0000","last_fullsized":"2023-07-06T01:19:26.631273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:32.163145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632124+0000","last_change":"2023-07-06T01:19:24.645784+0000","last_active":"2023-07-06T01:19:26.632124+0000","last_peered":"2023-07-06T01:19:26.632124+0000","last_clean":"2023-07-06T01:19:26.632124+0000","last_became_active":"2023-07-06T01:19:24.645447+0000","last_became_peered":"2023-07-06T01:19:24.645447+0000","last_unstale":"2023-07-06T01:19:26.632124+0000","last_undegraded":"2023-07-06T01:19:26.632124+0000","last_fullsized":"2023-07-06T01:19:26.632124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:08.965590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944733+0000","last_change":"2023-07-06T01:19:24.642392+0000","last_active":"2023-07-06T01:19:26.944733+0000","last_peered":"2023-07-06T01:19:26.944733+0000","last_clean":"2023-07-06T01:19:26.944733+0000","last_became_active":"2023-07-06T01:19:24.642122+0000","last_became_peered":"2023-07-06T01:19:24.642122+0000","last_unstale":"2023-07-06T01:19:26.944733+0000","last_undegraded":"2023-07-06T01:19:26.944733+0000","last_fullsized":"2023-07-06T01:19:26.944733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:01:58.134250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944450+0000","last_change":"2023-07-06T01:19:25.650007+0000","last_active":"2023-07-06T01:19:26.944450+0000","last_peered":"2023-07-06T01:19:26.944450+0000","last_clean":"2023-07-06T01:19:26.944450+0000","last_became_active":"2023-07-06T01:19:25.649108+0000","last_became_peered":"2023-07-06T01:19:25.649108+0000","last_unstale":"2023-07-06T01:19:26.944450+0000","last_undegraded":"2023-07-06T01:19:26.944450+0000","last_fullsized":"2023-07-06T01:19:26.944450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:22.147924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630751+0000","last_change":"2023-07-06T01:19:24.642694+0000","last_active":"2023-07-06T01:19:26.630751+0000","last_peered":"2023-07-06T01:19:26.630751+0000","last_clean":"2023-07-06T01:19:26.630751+0000","last_became_active":"2023-07-06T01:19:24.642130+0000","last_became_peered":"2023-07-06T01:19:24.642130+0000","last_unstale":"2023-07-06T01:19:26.630751+0000","last_undegraded":"2023-07-06T01:19:26.630751+0000","last_fullsized":"2023-07-06T01:19:26.630751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:09.755748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068713+0000","last_change":"2023-07-06T01:19:25.652413+0000","last_active":"2023-07-06T01:19:27.068713+0000","last_peered":"2023-07-06T01:19:27.068713+0000","last_clean":"2023-07-06T01:19:27.068713+0000","last_became_active":"2023-07-06T01:19:25.652051+0000","last_became_peered":"2023-07-06T01:19:25.652051+0000","last_unstale":"2023-07-06T01:19:27.068713+0000","last_undegraded":"2023-07-06T01:19:27.068713+0000","last_fullsized":"2023-07-06T01:19:27.068713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:44.291645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944695+0000","last_change":"2023-07-06T01:19:25.652994+0000","last_active":"2023-07-06T01:19:26.944695+0000","last_peered":"2023-07-06T01:19:26.944695+0000","last_clean":"2023-07-06T01:19:26.944695+0000","last_became_active":"2023-07-06T01:19:25.652484+0000","last_became_peered":"2023-07-06T01:19:25.652484+0000","last_unstale":"2023-07-06T01:19:26.944695+0000","last_undegraded":"2023-07-06T01:19:26.944695+0000","last_fullsized":"2023-07-06T01:19:26.944695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:14.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:19:24.621395+0000","last_change":"2023-07-06T01:19:24.621395+0000","last_active":"2023-07-06T01:19:24.621395+0000","last_peered":"2023-07-06T01:19:24.621395+0000","last_clean":"2023-07-06T01:19:24.621395+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:19:24.621395+0000","last_undegraded":"2023-07-06T01:19:24.621395+0000","last_fullsized":"2023-07-06T01:19:24.621395+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944583+0000","last_change":"2023-07-06T01:19:25.653857+0000","last_active":"2023-07-06T01:19:26.944583+0000","last_peered":"2023-07-06T01:19:26.944583+0000","last_clean":"2023-07-06T01:19:26.944583+0000","last_became_active":"2023-07-06T01:19:25.652976+0000","last_became_peered":"2023-07-06T01:19:25.652976+0000","last_unstale":"2023-07-06T01:19:26.944583+0000","last_undegraded":"2023-07-06T01:19:26.944583+0000","last_fullsized":"2023-07-06T01:19:26.944583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:25.724682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068930+0000","last_change":"2023-07-06T01:19:25.647146+0000","last_active":"2023-07-06T01:19:27.068930+0000","last_peered":"2023-07-06T01:19:27.068930+0000","last_clean":"2023-07-06T01:19:27.068930+0000","last_became_active":"2023-07-06T01:19:25.646863+0000","last_became_peered":"2023-07-06T01:19:25.646863+0000","last_unstale":"2023-07-06T01:19:27.068930+0000","last_undegraded":"2023-07-06T01:19:27.068930+0000","last_fullsized":"2023-07-06T01:19:27.068930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:21.566043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":88,"acting":88,"num_store_stats":6},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":44,"acting":44,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":757760,"data_stored":642336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27112,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344728,"statfs":{"total":96636764160,"available":96609001472,"internally_reserved":0,"allocated":163840,"data_stored":51968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":794624,"data_stored":662254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":78539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.248}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":78629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":79191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":200704,"data_stored":71886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:19:29.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:19:29.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.947+0000 7f20516d1700 1 Processor -- start 2023-07-06T01:19:29.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.947+0000 7f20516d1700 1 -- start start 2023-07-06T01:19:29.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f20516d1700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f204c0b36f0 0x7f204c0cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f20516d1700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f204c0d11a0 0x7f204c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f20516d1700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c0cebd0 con 0x7f204c0cf5a0 2023-07-06T01:19:29.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f20516d1700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204c15cff0 con 0x7f204c0d11a0 2023-07-06T01:19:29.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f20516d1700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204c0cdfc0 con 0x7f204c0b36f0 2023-07-06T01:19:29.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f204b7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f204c0d11a0 0x7f204c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f204a7fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f204c0cf5a0 0x7f204c0d0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36084/0 (socket says 172.21.15.22:36084) 2023-07-06T01:19:29.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f204b7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f204c0d11a0 0x7f204c0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45342/0 (socket says 172.21.15.22:45342) 2023-07-06T01:19:29.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f204a7fc700 1 -- 172.21.15.22:0/2596378675 learned_addr learned my addr 172.21.15.22:0/2596378675 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:29.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f204affd700 1 --2- 172.21.15.22:0/2596378675 >> v2:172.21.15.103:3300/0 conn(0x7f204c0b36f0 0x7f204c0cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f204b7fe700 1 -- 172.21.15.22:0/2596378675 >> v2:172.21.15.103:3300/0 conn(0x7f204c0b36f0 msgr2=0x7f204c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:29.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f204b7fe700 1 --2- 172.21.15.22:0/2596378675 >> v2:172.21.15.103:3300/0 conn(0x7f204c0b36f0 0x7f204c0cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f204b7fe700 1 -- 172.21.15.22:0/2596378675 >> v1:172.21.15.22:6789/0 conn(0x7f204c0cf5a0 legacy=0x7f204c0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f2049ffb700 1 -- 172.21.15.22:0/2596378675 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702438901 0 0) 0x7f204c0cebd0 con 0x7f204c0cf5a0 2023-07-06T01:19:29.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f204b7fe700 1 -- 172.21.15.22:0/2596378675 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c0ce2a0 con 0x7f204c0d11a0 2023-07-06T01:19:29.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.948+0000 7f204b7fe700 1 --2- 172.21.15.22:0/2596378675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f204c0d11a0 0x7f204c0b5da0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f204c08b8d0 tx=0x7f204400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2357f4df39bb8d03 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:29.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.949+0000 7f2049ffb700 1 -- 172.21.15.22:0/2596378675 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204400e070 con 0x7f204c0d11a0 2023-07-06T01:19:29.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.949+0000 7f2049ffb700 1 -- 172.21.15.22:0/2596378675 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204400bb20 con 0x7f204c0d11a0 2023-07-06T01:19:29.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.949+0000 7f2049ffb700 1 -- 172.21.15.22:0/2596378675 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044012430 con 0x7f204c0d11a0 2023-07-06T01:19:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.949+0000 7f20516d1700 1 -- 172.21.15.22:0/2596378675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f204c0d11a0 msgr2=0x7f204c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.949+0000 7f20516d1700 1 --2- 172.21.15.22:0/2596378675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f204c0d11a0 0x7f204c0b5da0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f204c08b8d0 tx=0x7f204400b330 comp rx=0 tx=0).stop 2023-07-06T01:19:29.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.949+0000 7f20516d1700 1 -- 172.21.15.22:0/2596378675 shutdown_connections 2023-07-06T01:19:29.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.949+0000 7f20516d1700 1 --2- 172.21.15.22:0/2596378675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f204c0d11a0 0x7f204c0b5da0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.949+0000 7f20516d1700 1 --2- 172.21.15.22:0/2596378675 >> v2:172.21.15.103:3300/0 conn(0x7f204c0b36f0 0x7f204c0cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.949+0000 7f20516d1700 1 -- 172.21.15.22:0/2596378675 >> 172.21.15.22:0/2596378675 conn(0x7f204c09c2e0 msgr2=0x7f204c0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:29.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.949+0000 7f20516d1700 1 -- 172.21.15.22:0/2596378675 shutdown_connections 2023-07-06T01:19:29.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.949+0000 7f20516d1700 1 -- 172.21.15.22:0/2596378675 wait complete. 2023-07-06T01:19:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.951+0000 7f20516d1700 1 Processor -- start 2023-07-06T01:19:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.951+0000 7f20516d1700 1 -- start start 2023-07-06T01:19:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.951+0000 7f20516d1700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f204c0b36f0 0x7f204c0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.951+0000 7f20516d1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f204c0cf5a0 0x7f204c0cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.951+0000 7f20516d1700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f204c0d11a0 0x7f204c0cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.951+0000 7f20516d1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c15f6a0 con 0x7f204c0cf5a0 2023-07-06T01:19:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.951+0000 7f20516d1700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c15ee00 con 0x7f204c0d11a0 2023-07-06T01:19:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.951+0000 7f20516d1700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c15f0e0 con 0x7f204c0b36f0 2023-07-06T01:19:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.951+0000 7f204a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f204c0cf5a0 0x7f204c0cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.951+0000 7f204a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f204c0cf5a0 0x7f204c0cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45346/0 (socket says 172.21.15.22:45346) 2023-07-06T01:19:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.951+0000 7f204a7fc700 1 -- 172.21.15.22:0/3618969405 learned_addr learned my addr 172.21.15.22:0/3618969405 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.952+0000 7f204b7fe700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f204c0d11a0 0x7f204c0cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.952+0000 7f204affd700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f204c0b36f0 0x7f204c0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.952+0000 7f204a7fc700 1 -- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f204c0b36f0 msgr2=0x7f204c0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.952+0000 7f204a7fc700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f204c0b36f0 0x7f204c0ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.952+0000 7f204a7fc700 1 -- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f204c0d11a0 msgr2=0x7f204c0cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.952+0000 7f204a7fc700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f204c0d11a0 0x7f204c0cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.952+0000 7f204a7fc700 1 -- 172.21.15.22:0/3618969405 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2044009a70 con 0x7f204c0cf5a0 2023-07-06T01:19:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.952+0000 7f204a7fc700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f204c0cf5a0 0x7f204c0cab90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f204000c540 tx=0x7f204000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.952+0000 7f20337fe700 1 -- 172.21.15.22:0/3618969405 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040004c80 con 0x7f204c0cf5a0 2023-07-06T01:19:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.952+0000 7f20337fe700 1 -- 172.21.15.22:0/3618969405 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2040004de0 con 0x7f204c0cf5a0 2023-07-06T01:19:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.952+0000 7f20337fe700 1 -- 172.21.15.22:0/3618969405 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040007310 con 0x7f204c0cf5a0 2023-07-06T01:19:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.952+0000 7f20516d1700 1 -- 172.21.15.22:0/3618969405 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c15f3c0 con 0x7f204c0cf5a0 2023-07-06T01:19:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.952+0000 7f20516d1700 1 -- 172.21.15.22:0/3618969405 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204c0cb610 con 0x7f204c0cf5a0 2023-07-06T01:19:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.953+0000 7f20516d1700 1 -- 172.21.15.22:0/3618969405 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204c0b4840 con 0x7f204c0cf5a0 2023-07-06T01:19:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.954+0000 7f20337fe700 1 -- 172.21.15.22:0/3618969405 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2040007470 con 0x7f204c0cf5a0 2023-07-06T01:19:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.954+0000 7f20337fe700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f203806ace0 0x7f203806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.954+0000 7f20337fe700 1 -- 172.21.15.22:0/3618969405 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2040089160 con 0x7f204c0cf5a0 2023-07-06T01:19:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.954+0000 7f204affd700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f203806ace0 0x7f203806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:29.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.955+0000 7f204affd700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f203806ace0 0x7f203806d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f203c009ab0 tx=0x7f203c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:29.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:29.958+0000 7f20337fe700 1 -- 172.21.15.22:0/3618969405 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f20400542a0 con 0x7f204c0cf5a0 2023-07-06T01:19:30.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.152+0000 7f20516d1700 1 -- 172.21.15.22:0/3618969405 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f204c0ce860 con 0x7f203806ace0 2023-07-06T01:19:30.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.161+0000 7f20337fe700 1 -- 172.21.15.22:0/3618969405 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286976 (secure 0 0 0) 0x7f203c0106e0 con 0x7f203806ace0 2023-07-06T01:19:30.168 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.164+0000 7f20516d1700 1 -- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f203806ace0 msgr2=0x7f203806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.164+0000 7f20516d1700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f203806ace0 0x7f203806d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f203c009ab0 tx=0x7f203c008040 comp rx=0 tx=0).stop 2023-07-06T01:19:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.164+0000 7f20516d1700 1 -- 172.21.15.22:0/3618969405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f204c0cf5a0 msgr2=0x7f204c0cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.164+0000 7f20516d1700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f204c0cf5a0 0x7f204c0cab90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f204000c540 tx=0x7f204000c900 comp rx=0 tx=0).stop 2023-07-06T01:19:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.164+0000 7f20516d1700 1 -- 172.21.15.22:0/3618969405 shutdown_connections 2023-07-06T01:19:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.164+0000 7f20516d1700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f203806ace0 0x7f203806d190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.164+0000 7f20516d1700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f204c0d11a0 0x7f204c0cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.164+0000 7f20516d1700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f204c0b36f0 0x7f204c0ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.164+0000 7f20516d1700 1 --2- 172.21.15.22:0/3618969405 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f204c0cf5a0 0x7f204c0cab90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.164+0000 7f20516d1700 1 -- 172.21.15.22:0/3618969405 >> 172.21.15.22:0/3618969405 conn(0x7f204c09c2e0 msgr2=0x7f204c0cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.164+0000 7f20516d1700 1 -- 172.21.15.22:0/3618969405 shutdown_connections 2023-07-06T01:19:30.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.164+0000 7f20516d1700 1 -- 172.21.15.22:0/3618969405 wait complete. 2023-07-06T01:19:30.172 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:19:30.179 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-07-06T01:19:29.998671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218480,"kb_used_data":2864,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756240,"statfs":{"total":773094113280,"available":772870389760,"internally_reserved":0,"allocated":2932736,"data_stored":1800466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001433"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068502+0000","last_change":"2023-07-06T01:19:25.653405+0000","last_active":"2023-07-06T01:19:27.068502+0000","last_peered":"2023-07-06T01:19:27.068502+0000","last_clean":"2023-07-06T01:19:27.068502+0000","last_became_active":"2023-07-06T01:19:25.653093+0000","last_became_peered":"2023-07-06T01:19:25.653093+0000","last_unstale":"2023-07-06T01:19:27.068502+0000","last_undegraded":"2023-07-06T01:19:27.068502+0000","last_fullsized":"2023-07-06T01:19:27.068502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:17.116328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632301+0000","last_change":"2023-07-06T01:19:25.649747+0000","last_active":"2023-07-06T01:19:26.632301+0000","last_peered":"2023-07-06T01:19:26.632301+0000","last_clean":"2023-07-06T01:19:26.632301+0000","last_became_active":"2023-07-06T01:19:25.649468+0000","last_became_peered":"2023-07-06T01:19:25.649468+0000","last_unstale":"2023-07-06T01:19:26.632301+0000","last_undegraded":"2023-07-06T01:19:26.632301+0000","last_fullsized":"2023-07-06T01:19:26.632301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:00.621635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632173+0000","last_change":"2023-07-06T01:19:25.660551+0000","last_active":"2023-07-06T01:19:26.632173+0000","last_peered":"2023-07-06T01:19:26.632173+0000","last_clean":"2023-07-06T01:19:26.632173+0000","last_became_active":"2023-07-06T01:19:25.660278+0000","last_became_peered":"2023-07-06T01:19:25.660278+0000","last_unstale":"2023-07-06T01:19:26.632173+0000","last_undegraded":"2023-07-06T01:19:26.632173+0000","last_fullsized":"2023-07-06T01:19:26.632173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:36.632721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068877+0000","last_change":"2023-07-06T01:19:25.647264+0000","last_active":"2023-07-06T01:19:27.068877+0000","last_peered":"2023-07-06T01:19:27.068877+0000","last_clean":"2023-07-06T01:19:27.068877+0000","last_became_active":"2023-07-06T01:19:25.646962+0000","last_became_peered":"2023-07-06T01:19:25.646962+0000","last_unstale":"2023-07-06T01:19:27.068877+0000","last_undegraded":"2023-07-06T01:19:27.068877+0000","last_fullsized":"2023-07-06T01:19:27.068877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:33.168360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631056+0000","last_change":"2023-07-06T01:19:25.655455+0000","last_active":"2023-07-06T01:19:26.631056+0000","last_peered":"2023-07-06T01:19:26.631056+0000","last_clean":"2023-07-06T01:19:26.631056+0000","last_became_active":"2023-07-06T01:19:25.655137+0000","last_became_peered":"2023-07-06T01:19:25.655137+0000","last_unstale":"2023-07-06T01:19:26.631056+0000","last_undegraded":"2023-07-06T01:19:26.631056+0000","last_fullsized":"2023-07-06T01:19:26.631056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:39.980297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631681+0000","last_change":"2023-07-06T01:19:25.660076+0000","last_active":"2023-07-06T01:19:26.631681+0000","last_peered":"2023-07-06T01:19:26.631681+0000","last_clean":"2023-07-06T01:19:26.631681+0000","last_became_active":"2023-07-06T01:19:25.659798+0000","last_became_peered":"2023-07-06T01:19:25.659798+0000","last_unstale":"2023-07-06T01:19:26.631681+0000","last_undegraded":"2023-07-06T01:19:26.631681+0000","last_fullsized":"2023-07-06T01:19:26.631681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:25.724682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068900+0000","last_change":"2023-07-06T01:19:25.649766+0000","last_active":"2023-07-06T01:19:27.068900+0000","last_peered":"2023-07-06T01:19:27.068900+0000","last_clean":"2023-07-06T01:19:27.068900+0000","last_became_active":"2023-07-06T01:19:25.649497+0000","last_became_peered":"2023-07-06T01:19:25.649497+0000","last_unstale":"2023-07-06T01:19:27.068900+0000","last_undegraded":"2023-07-06T01:19:27.068900+0000","last_fullsized":"2023-07-06T01:19:27.068900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:25.724682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631105+0000","last_change":"2023-07-06T01:19:25.657379+0000","last_active":"2023-07-06T01:19:26.631105+0000","last_peered":"2023-07-06T01:19:26.631105+0000","last_clean":"2023-07-06T01:19:26.631105+0000","last_became_active":"2023-07-06T01:19:25.657044+0000","last_became_peered":"2023-07-06T01:19:25.657044+0000","last_unstale":"2023-07-06T01:19:26.631105+0000","last_undegraded":"2023-07-06T01:19:26.631105+0000","last_fullsized":"2023-07-06T01:19:26.631105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:21:00.969228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630543+0000","last_change":"2023-07-06T01:19:25.654446+0000","last_active":"2023-07-06T01:19:26.630543+0000","last_peered":"2023-07-06T01:19:26.630543+0000","last_clean":"2023-07-06T01:19:26.630543+0000","last_became_active":"2023-07-06T01:19:25.653834+0000","last_became_peered":"2023-07-06T01:19:25.653834+0000","last_unstale":"2023-07-06T01:19:26.630543+0000","last_undegraded":"2023-07-06T01:19:26.630543+0000","last_fullsized":"2023-07-06T01:19:26.630543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:33.168360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632483+0000","last_change":"2023-07-06T01:19:25.648054+0000","last_active":"2023-07-06T01:19:26.632483+0000","last_peered":"2023-07-06T01:19:26.632483+0000","last_clean":"2023-07-06T01:19:26.632483+0000","last_became_active":"2023-07-06T01:19:25.647795+0000","last_became_peered":"2023-07-06T01:19:25.647795+0000","last_unstale":"2023-07-06T01:19:26.632483+0000","last_undegraded":"2023-07-06T01:19:26.632483+0000","last_fullsized":"2023-07-06T01:19:26.632483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632291+0000","last_change":"2023-07-06T01:19:25.657523+0000","last_active":"2023-07-06T01:19:26.632291+0000","last_peered":"2023-07-06T01:19:26.632291+0000","last_clean":"2023-07-06T01:19:26.632291+0000","last_became_active":"2023-07-06T01:19:25.657258+0000","last_became_peered":"2023-07-06T01:19:25.657258+0000","last_unstale":"2023-07-06T01:19:26.632291+0000","last_undegraded":"2023-07-06T01:19:26.632291+0000","last_fullsized":"2023-07-06T01:19:26.632291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:59.279465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068838+0000","last_change":"2023-07-06T01:19:25.660991+0000","last_active":"2023-07-06T01:19:27.068838+0000","last_peered":"2023-07-06T01:19:27.068838+0000","last_clean":"2023-07-06T01:19:27.068838+0000","last_became_active":"2023-07-06T01:19:25.660654+0000","last_became_peered":"2023-07-06T01:19:25.660654+0000","last_unstale":"2023-07-06T01:19:27.068838+0000","last_undegraded":"2023-07-06T01:19:27.068838+0000","last_fullsized":"2023-07-06T01:19:27.068838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:43:26.720400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068325+0000","last_change":"2023-07-06T01:19:25.654120+0000","last_active":"2023-07-06T01:19:27.068325+0000","last_peered":"2023-07-06T01:19:27.068325+0000","last_clean":"2023-07-06T01:19:27.068325+0000","last_became_active":"2023-07-06T01:19:25.653844+0000","last_became_peered":"2023-07-06T01:19:25.653844+0000","last_unstale":"2023-07-06T01:19:27.068325+0000","last_undegraded":"2023-07-06T01:19:27.068325+0000","last_fullsized":"2023-07-06T01:19:27.068325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:42:37.007207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631522+0000","last_change":"2023-07-06T01:19:25.650622+0000","last_active":"2023-07-06T01:19:26.631522+0000","last_peered":"2023-07-06T01:19:26.631522+0000","last_clean":"2023-07-06T01:19:26.631522+0000","last_became_active":"2023-07-06T01:19:25.650310+0000","last_became_peered":"2023-07-06T01:19:25.650310+0000","last_unstale":"2023-07-06T01:19:26.631522+0000","last_undegraded":"2023-07-06T01:19:26.631522+0000","last_fullsized":"2023-07-06T01:19:26.631522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:21.566043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069099+0000","last_change":"2023-07-06T01:19:25.651765+0000","last_active":"2023-07-06T01:19:27.069099+0000","last_peered":"2023-07-06T01:19:27.069099+0000","last_clean":"2023-07-06T01:19:27.069099+0000","last_became_active":"2023-07-06T01:19:25.651244+0000","last_became_peered":"2023-07-06T01:19:25.651244+0000","last_unstale":"2023-07-06T01:19:27.069099+0000","last_undegraded":"2023-07-06T01:19:27.069099+0000","last_fullsized":"2023-07-06T01:19:27.069099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:41:34.533716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631383+0000","last_change":"2023-07-06T01:19:25.663299+0000","last_active":"2023-07-06T01:19:26.631383+0000","last_peered":"2023-07-06T01:19:26.631383+0000","last_clean":"2023-07-06T01:19:26.631383+0000","last_became_active":"2023-07-06T01:19:25.663032+0000","last_became_peered":"2023-07-06T01:19:25.663032+0000","last_unstale":"2023-07-06T01:19:26.631383+0000","last_undegraded":"2023-07-06T01:19:26.631383+0000","last_fullsized":"2023-07-06T01:19:26.631383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:31.571390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630591+0000","last_change":"2023-07-06T01:19:25.654669+0000","last_active":"2023-07-06T01:19:26.630591+0000","last_peered":"2023-07-06T01:19:26.630591+0000","last_clean":"2023-07-06T01:19:26.630591+0000","last_became_active":"2023-07-06T01:19:25.654161+0000","last_became_peered":"2023-07-06T01:19:25.654161+0000","last_unstale":"2023-07-06T01:19:26.630591+0000","last_undegraded":"2023-07-06T01:19:26.630591+0000","last_fullsized":"2023-07-06T01:19:26.630591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:42.802038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631594+0000","last_change":"2023-07-06T01:19:25.651140+0000","last_active":"2023-07-06T01:19:26.631594+0000","last_peered":"2023-07-06T01:19:26.631594+0000","last_clean":"2023-07-06T01:19:26.631594+0000","last_became_active":"2023-07-06T01:19:25.650810+0000","last_became_peered":"2023-07-06T01:19:25.650810+0000","last_unstale":"2023-07-06T01:19:26.631594+0000","last_undegraded":"2023-07-06T01:19:26.631594+0000","last_fullsized":"2023-07-06T01:19:26.631594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:39.980297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068611+0000","last_change":"2023-07-06T01:19:25.645030+0000","last_active":"2023-07-06T01:19:27.068611+0000","last_peered":"2023-07-06T01:19:27.068611+0000","last_clean":"2023-07-06T01:19:27.068611+0000","last_became_active":"2023-07-06T01:19:25.644689+0000","last_became_peered":"2023-07-06T01:19:25.644689+0000","last_unstale":"2023-07-06T01:19:27.068611+0000","last_undegraded":"2023-07-06T01:19:27.068611+0000","last_fullsized":"2023-07-06T01:19:27.068611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:14.823559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632289+0000","last_change":"2023-07-06T01:19:25.660308+0000","last_active":"2023-07-06T01:19:26.632289+0000","last_peered":"2023-07-06T01:19:26.632289+0000","last_clean":"2023-07-06T01:19:26.632289+0000","last_became_active":"2023-07-06T01:19:25.658245+0000","last_became_peered":"2023-07-06T01:19:25.658245+0000","last_unstale":"2023-07-06T01:19:26.632289+0000","last_undegraded":"2023-07-06T01:19:26.632289+0000","last_fullsized":"2023-07-06T01:19:26.632289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:14.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069296+0000","last_change":"2023-07-06T01:19:25.660004+0000","last_active":"2023-07-06T01:19:27.069296+0000","last_peered":"2023-07-06T01:19:27.069296+0000","last_clean":"2023-07-06T01:19:27.069296+0000","last_became_active":"2023-07-06T01:19:25.659487+0000","last_became_peered":"2023-07-06T01:19:25.659487+0000","last_unstale":"2023-07-06T01:19:27.069296+0000","last_undegraded":"2023-07-06T01:19:27.069296+0000","last_fullsized":"2023-07-06T01:19:27.069296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:14.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630591+0000","last_change":"2023-07-06T01:19:25.636207+0000","last_active":"2023-07-06T01:19:26.630591+0000","last_peered":"2023-07-06T01:19:26.630591+0000","last_clean":"2023-07-06T01:19:26.630591+0000","last_became_active":"2023-07-06T01:19:25.635876+0000","last_became_peered":"2023-07-06T01:19:25.635876+0000","last_unstale":"2023-07-06T01:19:26.630591+0000","last_undegraded":"2023-07-06T01:19:26.630591+0000","last_fullsized":"2023-07-06T01:19:26.630591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:25.693365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944279+0000","last_change":"2023-07-06T01:19:25.655490+0000","last_active":"2023-07-06T01:19:26.944279+0000","last_peered":"2023-07-06T01:19:26.944279+0000","last_clean":"2023-07-06T01:19:26.944279+0000","last_became_active":"2023-07-06T01:19:25.655137+0000","last_became_peered":"2023-07-06T01:19:25.655137+0000","last_unstale":"2023-07-06T01:19:26.944279+0000","last_undegraded":"2023-07-06T01:19:26.944279+0000","last_fullsized":"2023-07-06T01:19:26.944279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:09.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068827+0000","last_change":"2023-07-06T01:19:25.655082+0000","last_active":"2023-07-06T01:19:27.068827+0000","last_peered":"2023-07-06T01:19:27.068827+0000","last_clean":"2023-07-06T01:19:27.068827+0000","last_became_active":"2023-07-06T01:19:25.654779+0000","last_became_peered":"2023-07-06T01:19:25.654779+0000","last_unstale":"2023-07-06T01:19:27.068827+0000","last_undegraded":"2023-07-06T01:19:27.068827+0000","last_fullsized":"2023-07-06T01:19:27.068827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:22.147924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068831+0000","last_change":"2023-07-06T01:19:25.644786+0000","last_active":"2023-07-06T01:19:27.068831+0000","last_peered":"2023-07-06T01:19:27.068831+0000","last_clean":"2023-07-06T01:19:27.068831+0000","last_became_active":"2023-07-06T01:19:25.644482+0000","last_became_peered":"2023-07-06T01:19:25.644482+0000","last_unstale":"2023-07-06T01:19:27.068831+0000","last_undegraded":"2023-07-06T01:19:27.068831+0000","last_fullsized":"2023-07-06T01:19:27.068831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:42.802038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068413+0000","last_change":"2023-07-06T01:19:25.642117+0000","last_active":"2023-07-06T01:19:27.068413+0000","last_peered":"2023-07-06T01:19:27.068413+0000","last_clean":"2023-07-06T01:19:27.068413+0000","last_became_active":"2023-07-06T01:19:25.641830+0000","last_became_peered":"2023-07-06T01:19:25.641830+0000","last_unstale":"2023-07-06T01:19:27.068413+0000","last_undegraded":"2023-07-06T01:19:27.068413+0000","last_fullsized":"2023-07-06T01:19:27.068413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:43:11.310891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631861+0000","last_change":"2023-07-06T01:19:25.656605+0000","last_active":"2023-07-06T01:19:26.631861+0000","last_peered":"2023-07-06T01:19:26.631861+0000","last_clean":"2023-07-06T01:19:26.631861+0000","last_became_active":"2023-07-06T01:19:25.656259+0000","last_became_peered":"2023-07-06T01:19:25.656259+0000","last_unstale":"2023-07-06T01:19:26.631861+0000","last_undegraded":"2023-07-06T01:19:26.631861+0000","last_fullsized":"2023-07-06T01:19:26.631861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:40.213207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631966+0000","last_change":"2023-07-06T01:19:24.646013+0000","last_active":"2023-07-06T01:19:26.631966+0000","last_peered":"2023-07-06T01:19:26.631966+0000","last_clean":"2023-07-06T01:19:26.631966+0000","last_became_active":"2023-07-06T01:19:24.644677+0000","last_became_peered":"2023-07-06T01:19:24.644677+0000","last_unstale":"2023-07-06T01:19:26.631966+0000","last_undegraded":"2023-07-06T01:19:26.631966+0000","last_fullsized":"2023-07-06T01:19:26.631966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:17.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944791+0000","last_change":"2023-07-06T01:19:25.650690+0000","last_active":"2023-07-06T01:19:26.944791+0000","last_peered":"2023-07-06T01:19:26.944791+0000","last_clean":"2023-07-06T01:19:26.944791+0000","last_became_active":"2023-07-06T01:19:25.650316+0000","last_became_peered":"2023-07-06T01:19:25.650316+0000","last_unstale":"2023-07-06T01:19:26.944791+0000","last_undegraded":"2023-07-06T01:19:26.944791+0000","last_fullsized":"2023-07-06T01:19:26.944791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:02.031653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068547+0000","last_change":"2023-07-06T01:19:24.645053+0000","last_active":"2023-07-06T01:19:27.068547+0000","last_peered":"2023-07-06T01:19:27.068547+0000","last_clean":"2023-07-06T01:19:27.068547+0000","last_became_active":"2023-07-06T01:19:24.644766+0000","last_became_peered":"2023-07-06T01:19:24.644766+0000","last_unstale":"2023-07-06T01:19:27.068547+0000","last_undegraded":"2023-07-06T01:19:27.068547+0000","last_fullsized":"2023-07-06T01:19:27.068547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:43.286430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630830+0000","last_change":"2023-07-06T01:19:25.649424+0000","last_active":"2023-07-06T01:19:26.630830+0000","last_peered":"2023-07-06T01:19:26.630830+0000","last_clean":"2023-07-06T01:19:26.630830+0000","last_became_active":"2023-07-06T01:19:25.649164+0000","last_became_peered":"2023-07-06T01:19:25.649164+0000","last_unstale":"2023-07-06T01:19:26.630830+0000","last_undegraded":"2023-07-06T01:19:26.630830+0000","last_fullsized":"2023-07-06T01:19:26.630830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:00.621635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944431+0000","last_change":"2023-07-06T01:19:24.645861+0000","last_active":"2023-07-06T01:19:26.944431+0000","last_peered":"2023-07-06T01:19:26.944431+0000","last_clean":"2023-07-06T01:19:26.944431+0000","last_became_active":"2023-07-06T01:19:24.645583+0000","last_became_peered":"2023-07-06T01:19:24.645583+0000","last_unstale":"2023-07-06T01:19:26.944431+0000","last_undegraded":"2023-07-06T01:19:26.944431+0000","last_fullsized":"2023-07-06T01:19:26.944431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:38.975082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069581+0000","last_change":"2023-07-06T01:19:25.656345+0000","last_active":"2023-07-06T01:19:27.069581+0000","last_peered":"2023-07-06T01:19:27.069581+0000","last_clean":"2023-07-06T01:19:27.069581+0000","last_became_active":"2023-07-06T01:19:25.655955+0000","last_became_peered":"2023-07-06T01:19:25.655955+0000","last_unstale":"2023-07-06T01:19:27.069581+0000","last_undegraded":"2023-07-06T01:19:27.069581+0000","last_fullsized":"2023-07-06T01:19:27.069581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:45.855998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630810+0000","last_change":"2023-07-06T01:19:24.643151+0000","last_active":"2023-07-06T01:19:26.630810+0000","last_peered":"2023-07-06T01:19:26.630810+0000","last_clean":"2023-07-06T01:19:26.630810+0000","last_became_active":"2023-07-06T01:19:24.642827+0000","last_became_peered":"2023-07-06T01:19:24.642827+0000","last_unstale":"2023-07-06T01:19:26.630810+0000","last_undegraded":"2023-07-06T01:19:26.630810+0000","last_fullsized":"2023-07-06T01:19:26.630810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:08.965590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944402+0000","last_change":"2023-07-06T01:19:25.652005+0000","last_active":"2023-07-06T01:19:26.944402+0000","last_peered":"2023-07-06T01:19:26.944402+0000","last_clean":"2023-07-06T01:19:26.944402+0000","last_became_active":"2023-07-06T01:19:25.651299+0000","last_became_peered":"2023-07-06T01:19:25.651299+0000","last_unstale":"2023-07-06T01:19:26.944402+0000","last_undegraded":"2023-07-06T01:19:26.944402+0000","last_fullsized":"2023-07-06T01:19:26.944402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632387+0000","last_change":"2023-07-06T01:19:24.642960+0000","last_active":"2023-07-06T01:19:26.632387+0000","last_peered":"2023-07-06T01:19:26.632387+0000","last_clean":"2023-07-06T01:19:26.632387+0000","last_became_active":"2023-07-06T01:19:24.642667+0000","last_became_peered":"2023-07-06T01:19:24.642667+0000","last_unstale":"2023-07-06T01:19:26.632387+0000","last_undegraded":"2023-07-06T01:19:26.632387+0000","last_fullsized":"2023-07-06T01:19:26.632387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:30.566175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944596+0000","last_change":"2023-07-06T01:19:25.650426+0000","last_active":"2023-07-06T01:19:26.944596+0000","last_peered":"2023-07-06T01:19:26.944596+0000","last_clean":"2023-07-06T01:19:26.944596+0000","last_became_active":"2023-07-06T01:19:25.650130+0000","last_became_peered":"2023-07-06T01:19:25.650130+0000","last_unstale":"2023-07-06T01:19:26.944596+0000","last_undegraded":"2023-07-06T01:19:26.944596+0000","last_fullsized":"2023-07-06T01:19:26.944596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:45:08.937322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069529+0000","last_change":"2023-07-06T01:19:24.644753+0000","last_active":"2023-07-06T01:19:27.069529+0000","last_peered":"2023-07-06T01:19:27.069529+0000","last_clean":"2023-07-06T01:19:27.069529+0000","last_became_active":"2023-07-06T01:19:24.644420+0000","last_became_peered":"2023-07-06T01:19:24.644420+0000","last_unstale":"2023-07-06T01:19:27.069529+0000","last_undegraded":"2023-07-06T01:19:27.069529+0000","last_fullsized":"2023-07-06T01:19:27.069529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:01.026438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068483+0000","last_change":"2023-07-06T01:19:25.652623+0000","last_active":"2023-07-06T01:19:27.068483+0000","last_peered":"2023-07-06T01:19:27.068483+0000","last_clean":"2023-07-06T01:19:27.068483+0000","last_became_active":"2023-07-06T01:19:25.652341+0000","last_became_peered":"2023-07-06T01:19:25.652341+0000","last_unstale":"2023-07-06T01:19:27.068483+0000","last_undegraded":"2023-07-06T01:19:27.068483+0000","last_fullsized":"2023-07-06T01:19:27.068483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:21:00.969228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632583+0000","last_change":"2023-07-06T01:19:24.648863+0000","last_active":"2023-07-06T01:19:26.632583+0000","last_peered":"2023-07-06T01:19:26.632583+0000","last_clean":"2023-07-06T01:19:26.632583+0000","last_became_active":"2023-07-06T01:19:24.648520+0000","last_became_peered":"2023-07-06T01:19:24.648520+0000","last_unstale":"2023-07-06T01:19:26.632583+0000","last_undegraded":"2023-07-06T01:19:26.632583+0000","last_fullsized":"2023-07-06T01:19:26.632583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:31:21.451680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632381+0000","last_change":"2023-07-06T01:19:25.653993+0000","last_active":"2023-07-06T01:19:26.632381+0000","last_peered":"2023-07-06T01:19:26.632381+0000","last_clean":"2023-07-06T01:19:26.632381+0000","last_became_active":"2023-07-06T01:19:25.653093+0000","last_became_peered":"2023-07-06T01:19:25.653093+0000","last_unstale":"2023-07-06T01:19:26.632381+0000","last_undegraded":"2023-07-06T01:19:26.632381+0000","last_fullsized":"2023-07-06T01:19:26.632381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:02.031653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069278+0000","last_change":"2023-07-06T01:19:24.651048+0000","last_active":"2023-07-06T01:19:27.069278+0000","last_peered":"2023-07-06T01:19:27.069278+0000","last_clean":"2023-07-06T01:19:27.069278+0000","last_became_active":"2023-07-06T01:19:24.650735+0000","last_became_peered":"2023-07-06T01:19:24.650735+0000","last_unstale":"2023-07-06T01:19:27.069278+0000","last_undegraded":"2023-07-06T01:19:27.069278+0000","last_fullsized":"2023-07-06T01:19:27.069278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:58.274250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944680+0000","last_change":"2023-07-06T01:19:25.651034+0000","last_active":"2023-07-06T01:19:26.944680+0000","last_peered":"2023-07-06T01:19:26.944680+0000","last_clean":"2023-07-06T01:19:26.944680+0000","last_became_active":"2023-07-06T01:19:25.650753+0000","last_became_peered":"2023-07-06T01:19:25.650753+0000","last_unstale":"2023-07-06T01:19:26.944680+0000","last_undegraded":"2023-07-06T01:19:26.944680+0000","last_fullsized":"2023-07-06T01:19:26.944680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:36.632721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630793+0000","last_change":"2023-07-06T01:19:24.648974+0000","last_active":"2023-07-06T01:19:26.630793+0000","last_peered":"2023-07-06T01:19:26.630793+0000","last_clean":"2023-07-06T01:19:26.630793+0000","last_became_active":"2023-07-06T01:19:24.648691+0000","last_became_peered":"2023-07-06T01:19:24.648691+0000","last_unstale":"2023-07-06T01:19:26.630793+0000","last_undegraded":"2023-07-06T01:19:26.630793+0000","last_fullsized":"2023-07-06T01:19:26.630793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:44.850783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631977+0000","last_change":"2023-07-06T01:19:25.656530+0000","last_active":"2023-07-06T01:19:26.631977+0000","last_peered":"2023-07-06T01:19:26.631977+0000","last_clean":"2023-07-06T01:19:26.631977+0000","last_became_active":"2023-07-06T01:19:25.655966+0000","last_became_peered":"2023-07-06T01:19:25.655966+0000","last_unstale":"2023-07-06T01:19:26.631977+0000","last_undegraded":"2023-07-06T01:19:26.631977+0000","last_fullsized":"2023-07-06T01:19:26.631977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:10.760963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069121+0000","last_change":"2023-07-06T01:19:24.649133+0000","last_active":"2023-07-06T01:19:27.069121+0000","last_peered":"2023-07-06T01:19:27.069121+0000","last_clean":"2023-07-06T01:19:27.069121+0000","last_became_active":"2023-07-06T01:19:24.648818+0000","last_became_peered":"2023-07-06T01:19:24.648818+0000","last_unstale":"2023-07-06T01:19:27.069121+0000","last_undegraded":"2023-07-06T01:19:27.069121+0000","last_fullsized":"2023-07-06T01:19:27.069121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:45:07.932107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630689+0000","last_change":"2023-07-06T01:19:25.646210+0000","last_active":"2023-07-06T01:19:26.630689+0000","last_peered":"2023-07-06T01:19:26.630689+0000","last_clean":"2023-07-06T01:19:26.630689+0000","last_became_active":"2023-07-06T01:19:25.645912+0000","last_became_peered":"2023-07-06T01:19:25.645912+0000","last_unstale":"2023-07-06T01:19:26.630689+0000","last_undegraded":"2023-07-06T01:19:26.630689+0000","last_fullsized":"2023-07-06T01:19:26.630689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:21.566043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068646+0000","last_change":"2023-07-06T01:19:24.642544+0000","last_active":"2023-07-06T01:19:27.068646+0000","last_peered":"2023-07-06T01:19:27.068646+0000","last_clean":"2023-07-06T01:19:27.068646+0000","last_became_active":"2023-07-06T01:19:24.642267+0000","last_became_peered":"2023-07-06T01:19:24.642267+0000","last_unstale":"2023-07-06T01:19:27.068646+0000","last_undegraded":"2023-07-06T01:19:27.068646+0000","last_fullsized":"2023-07-06T01:19:27.068646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:20:59.964013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944798+0000","last_change":"2023-07-06T01:19:25.650388+0000","last_active":"2023-07-06T01:19:26.944798+0000","last_peered":"2023-07-06T01:19:26.944798+0000","last_clean":"2023-07-06T01:19:26.944798+0000","last_became_active":"2023-07-06T01:19:25.650047+0000","last_became_peered":"2023-07-06T01:19:25.650047+0000","last_unstale":"2023-07-06T01:19:26.944798+0000","last_undegraded":"2023-07-06T01:19:26.944798+0000","last_fullsized":"2023-07-06T01:19:26.944798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:08:02.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069262+0000","last_change":"2023-07-06T01:19:24.647357+0000","last_active":"2023-07-06T01:19:27.069262+0000","last_peered":"2023-07-06T01:19:27.069262+0000","last_clean":"2023-07-06T01:19:27.069262+0000","last_became_active":"2023-07-06T01:19:24.647103+0000","last_became_peered":"2023-07-06T01:19:24.647103+0000","last_unstale":"2023-07-06T01:19:27.069262+0000","last_undegraded":"2023-07-06T01:19:27.069262+0000","last_fullsized":"2023-07-06T01:19:27.069262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:35.627505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631268+0000","last_change":"2023-07-06T01:19:25.649637+0000","last_active":"2023-07-06T01:19:26.631268+0000","last_peered":"2023-07-06T01:19:26.631268+0000","last_clean":"2023-07-06T01:19:26.631268+0000","last_became_active":"2023-07-06T01:19:25.649363+0000","last_became_peered":"2023-07-06T01:19:25.649363+0000","last_unstale":"2023-07-06T01:19:26.631268+0000","last_undegraded":"2023-07-06T01:19:26.631268+0000","last_fullsized":"2023-07-06T01:19:26.631268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:44.291645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069101+0000","last_change":"2023-07-06T01:19:24.647245+0000","last_active":"2023-07-06T01:19:27.069101+0000","last_peered":"2023-07-06T01:19:27.069101+0000","last_clean":"2023-07-06T01:19:27.069101+0000","last_became_active":"2023-07-06T01:19:24.646993+0000","last_became_peered":"2023-07-06T01:19:24.646993+0000","last_unstale":"2023-07-06T01:19:27.069101+0000","last_undegraded":"2023-07-06T01:19:27.069101+0000","last_fullsized":"2023-07-06T01:19:27.069101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:17.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631382+0000","last_change":"2023-07-06T01:19:25.657736+0000","last_active":"2023-07-06T01:19:26.631382+0000","last_peered":"2023-07-06T01:19:26.631382+0000","last_clean":"2023-07-06T01:19:26.631382+0000","last_became_active":"2023-07-06T01:19:25.657511+0000","last_became_peered":"2023-07-06T01:19:25.657511+0000","last_unstale":"2023-07-06T01:19:26.631382+0000","last_undegraded":"2023-07-06T01:19:26.631382+0000","last_fullsized":"2023-07-06T01:19:26.631382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:00.621635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069462+0000","last_change":"2023-07-06T01:19:24.647904+0000","last_active":"2023-07-06T01:19:27.069462+0000","last_peered":"2023-07-06T01:19:27.069462+0000","last_clean":"2023-07-06T01:19:27.069462+0000","last_became_active":"2023-07-06T01:19:24.647621+0000","last_became_peered":"2023-07-06T01:19:24.647621+0000","last_unstale":"2023-07-06T01:19:27.069462+0000","last_undegraded":"2023-07-06T01:19:27.069462+0000","last_fullsized":"2023-07-06T01:19:27.069462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:08:01.460887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069126+0000","last_change":"2023-07-06T01:19:25.652728+0000","last_active":"2023-07-06T01:19:27.069126+0000","last_peered":"2023-07-06T01:19:27.069126+0000","last_clean":"2023-07-06T01:19:27.069126+0000","last_became_active":"2023-07-06T01:19:25.652420+0000","last_became_peered":"2023-07-06T01:19:25.652420+0000","last_unstale":"2023-07-06T01:19:27.069126+0000","last_undegraded":"2023-07-06T01:19:27.069126+0000","last_fullsized":"2023-07-06T01:19:27.069126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:40.213207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631482+0000","last_change":"2023-07-06T01:19:24.640722+0000","last_active":"2023-07-06T01:19:26.631482+0000","last_peered":"2023-07-06T01:19:26.631482+0000","last_clean":"2023-07-06T01:19:26.631482+0000","last_became_active":"2023-07-06T01:19:24.640417+0000","last_became_peered":"2023-07-06T01:19:24.640417+0000","last_unstale":"2023-07-06T01:19:26.631482+0000","last_undegraded":"2023-07-06T01:19:26.631482+0000","last_fullsized":"2023-07-06T01:19:26.631482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:24.688149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630955+0000","last_change":"2023-07-06T01:19:25.636624+0000","last_active":"2023-07-06T01:19:26.630955+0000","last_peered":"2023-07-06T01:19:26.630955+0000","last_clean":"2023-07-06T01:19:26.630955+0000","last_became_active":"2023-07-06T01:19:25.636320+0000","last_became_peered":"2023-07-06T01:19:25.636320+0000","last_unstale":"2023-07-06T01:19:26.630955+0000","last_undegraded":"2023-07-06T01:19:26.630955+0000","last_fullsized":"2023-07-06T01:19:26.630955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944677+0000","last_change":"2023-07-06T01:19:24.645728+0000","last_active":"2023-07-06T01:19:26.944677+0000","last_peered":"2023-07-06T01:19:26.944677+0000","last_clean":"2023-07-06T01:19:26.944677+0000","last_became_active":"2023-07-06T01:19:24.644479+0000","last_became_peered":"2023-07-06T01:19:24.644479+0000","last_unstale":"2023-07-06T01:19:26.944677+0000","last_undegraded":"2023-07-06T01:19:26.944677+0000","last_fullsized":"2023-07-06T01:19:26.944677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:58.274250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632444+0000","last_change":"2023-07-06T01:19:25.658759+0000","last_active":"2023-07-06T01:19:26.632444+0000","last_peered":"2023-07-06T01:19:26.632444+0000","last_clean":"2023-07-06T01:19:26.632444+0000","last_became_active":"2023-07-06T01:19:25.658482+0000","last_became_peered":"2023-07-06T01:19:25.658482+0000","last_unstale":"2023-07-06T01:19:26.632444+0000","last_undegraded":"2023-07-06T01:19:26.632444+0000","last_fullsized":"2023-07-06T01:19:26.632444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:58:41.885622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069265+0000","last_change":"2023-07-06T01:19:24.649526+0000","last_active":"2023-07-06T01:19:27.069265+0000","last_peered":"2023-07-06T01:19:27.069265+0000","last_clean":"2023-07-06T01:19:27.069265+0000","last_became_active":"2023-07-06T01:19:24.648970+0000","last_became_peered":"2023-07-06T01:19:24.648970+0000","last_unstale":"2023-07-06T01:19:27.069265+0000","last_undegraded":"2023-07-06T01:19:27.069265+0000","last_fullsized":"2023-07-06T01:19:27.069265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:41.796823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632518+0000","last_change":"2023-07-06T01:19:25.648610+0000","last_active":"2023-07-06T01:19:26.632518+0000","last_peered":"2023-07-06T01:19:26.632518+0000","last_clean":"2023-07-06T01:19:26.632518+0000","last_became_active":"2023-07-06T01:19:25.648317+0000","last_became_peered":"2023-07-06T01:19:25.648317+0000","last_unstale":"2023-07-06T01:19:26.632518+0000","last_undegraded":"2023-07-06T01:19:26.632518+0000","last_fullsized":"2023-07-06T01:19:26.632518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:47:36.660557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631774+0000","last_change":"2023-07-06T01:19:24.641218+0000","last_active":"2023-07-06T01:19:26.631774+0000","last_peered":"2023-07-06T01:19:26.631774+0000","last_clean":"2023-07-06T01:19:26.631774+0000","last_became_active":"2023-07-06T01:19:24.640780+0000","last_became_peered":"2023-07-06T01:19:24.640780+0000","last_unstale":"2023-07-06T01:19:26.631774+0000","last_undegraded":"2023-07-06T01:19:26.631774+0000","last_fullsized":"2023-07-06T01:19:26.631774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:21.142709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631764+0000","last_change":"2023-07-06T01:19:25.657935+0000","last_active":"2023-07-06T01:19:26.631764+0000","last_peered":"2023-07-06T01:19:26.631764+0000","last_clean":"2023-07-06T01:19:26.631764+0000","last_became_active":"2023-07-06T01:19:25.653472+0000","last_became_peered":"2023-07-06T01:19:25.653472+0000","last_unstale":"2023-07-06T01:19:26.631764+0000","last_undegraded":"2023-07-06T01:19:26.631764+0000","last_fullsized":"2023-07-06T01:19:26.631764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:45:08.937322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944291+0000","last_change":"2023-07-06T01:19:24.645128+0000","last_active":"2023-07-06T01:19:26.944291+0000","last_peered":"2023-07-06T01:19:26.944291+0000","last_clean":"2023-07-06T01:19:26.944291+0000","last_became_active":"2023-07-06T01:19:24.644825+0000","last_became_peered":"2023-07-06T01:19:24.644825+0000","last_unstale":"2023-07-06T01:19:26.944291+0000","last_undegraded":"2023-07-06T01:19:26.944291+0000","last_fullsized":"2023-07-06T01:19:26.944291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:14:59.616420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069000+0000","last_change":"2023-07-06T01:19:25.654566+0000","last_active":"2023-07-06T01:19:27.069000+0000","last_peered":"2023-07-06T01:19:27.069000+0000","last_clean":"2023-07-06T01:19:27.069000+0000","last_became_active":"2023-07-06T01:19:25.654238+0000","last_became_peered":"2023-07-06T01:19:25.654238+0000","last_unstale":"2023-07-06T01:19:27.069000+0000","last_undegraded":"2023-07-06T01:19:27.069000+0000","last_fullsized":"2023-07-06T01:19:27.069000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:45.855998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944614+0000","last_change":"2023-07-06T01:19:24.648921+0000","last_active":"2023-07-06T01:19:26.944614+0000","last_peered":"2023-07-06T01:19:26.944614+0000","last_clean":"2023-07-06T01:19:26.944614+0000","last_became_active":"2023-07-06T01:19:24.648676+0000","last_became_peered":"2023-07-06T01:19:24.648676+0000","last_unstale":"2023-07-06T01:19:26.944614+0000","last_undegraded":"2023-07-06T01:19:26.944614+0000","last_fullsized":"2023-07-06T01:19:26.944614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:41:33.528500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632266+0000","last_change":"2023-07-06T01:19:20.074195+0000","last_active":"2023-07-06T01:19:26.632266+0000","last_peered":"2023-07-06T01:19:26.632266+0000","last_clean":"2023-07-06T01:19:26.632266+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:19:26.632266+0000","last_undegraded":"2023-07-06T01:19:26.632266+0000","last_fullsized":"2023-07-06T01:19:26.632266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:07:56.411730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944214+0000","last_change":"2023-07-06T01:19:25.652132+0000","last_active":"2023-07-06T01:19:26.944214+0000","last_peered":"2023-07-06T01:19:26.944214+0000","last_clean":"2023-07-06T01:19:26.944214+0000","last_became_active":"2023-07-06T01:19:25.651678+0000","last_became_peered":"2023-07-06T01:19:25.651678+0000","last_unstale":"2023-07-06T01:19:26.944214+0000","last_undegraded":"2023-07-06T01:19:26.944214+0000","last_fullsized":"2023-07-06T01:19:26.944214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:45.855998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944468+0000","last_change":"2023-07-06T01:19:19.587110+0000","last_active":"2023-07-06T01:19:26.944468+0000","last_peered":"2023-07-06T01:19:26.944468+0000","last_clean":"2023-07-06T01:19:26.944468+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:19:26.944468+0000","last_undegraded":"2023-07-06T01:19:26.944468+0000","last_fullsized":"2023-07-06T01:19:26.944468+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:12.165222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630881+0000","last_change":"2023-07-06T01:19:24.640259+0000","last_active":"2023-07-06T01:19:26.630881+0000","last_peered":"2023-07-06T01:19:26.630881+0000","last_clean":"2023-07-06T01:19:26.630881+0000","last_became_active":"2023-07-06T01:19:24.639942+0000","last_became_peered":"2023-07-06T01:19:24.639942+0000","last_unstale":"2023-07-06T01:19:26.630881+0000","last_undegraded":"2023-07-06T01:19:26.630881+0000","last_fullsized":"2023-07-06T01:19:26.630881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:30.566175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631522+0000","last_change":"2023-07-06T01:19:25.657104+0000","last_active":"2023-07-06T01:19:26.631522+0000","last_peered":"2023-07-06T01:19:26.631522+0000","last_clean":"2023-07-06T01:19:26.631522+0000","last_became_active":"2023-07-06T01:19:25.656804+0000","last_became_peered":"2023-07-06T01:19:25.656804+0000","last_unstale":"2023-07-06T01:19:26.631522+0000","last_undegraded":"2023-07-06T01:19:26.631522+0000","last_fullsized":"2023-07-06T01:19:26.631522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068648+0000","last_change":"2023-07-06T01:19:19.584242+0000","last_active":"2023-07-06T01:19:27.068648+0000","last_peered":"2023-07-06T01:19:27.068648+0000","last_clean":"2023-07-06T01:19:27.068648+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:19:27.068648+0000","last_undegraded":"2023-07-06T01:19:27.068648+0000","last_fullsized":"2023-07-06T01:19:27.068648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:27.113987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":169,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068934+0000","last_change":"2023-07-06T01:19:19.586495+0000","last_active":"2023-07-06T01:19:27.068934+0000","last_peered":"2023-07-06T01:19:27.068934+0000","last_clean":"2023-07-06T01:19:27.068934+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:19:27.068934+0000","last_undegraded":"2023-07-06T01:19:27.068934+0000","last_fullsized":"2023-07-06T01:19:27.068934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:01:53.085093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630456+0000","last_change":"2023-07-06T01:19:24.642436+0000","last_active":"2023-07-06T01:19:26.630456+0000","last_peered":"2023-07-06T01:19:26.630456+0000","last_clean":"2023-07-06T01:19:26.630456+0000","last_became_active":"2023-07-06T01:19:24.641979+0000","last_became_peered":"2023-07-06T01:19:24.641979+0000","last_unstale":"2023-07-06T01:19:26.630456+0000","last_undegraded":"2023-07-06T01:19:26.630456+0000","last_fullsized":"2023-07-06T01:19:26.630456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:43.286430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630728+0000","last_change":"2023-07-06T01:19:25.639313+0000","last_active":"2023-07-06T01:19:26.630728+0000","last_peered":"2023-07-06T01:19:26.630728+0000","last_clean":"2023-07-06T01:19:26.630728+0000","last_became_active":"2023-07-06T01:19:25.639042+0000","last_became_peered":"2023-07-06T01:19:25.639042+0000","last_unstale":"2023-07-06T01:19:26.630728+0000","last_undegraded":"2023-07-06T01:19:26.630728+0000","last_fullsized":"2023-07-06T01:19:26.630728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:40.213207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068698+0000","last_change":"2023-07-06T01:19:19.586426+0000","last_active":"2023-07-06T01:19:27.068698+0000","last_peered":"2023-07-06T01:19:27.068698+0000","last_clean":"2023-07-06T01:19:27.068698+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:19:27.068698+0000","last_undegraded":"2023-07-06T01:19:27.068698+0000","last_fullsized":"2023-07-06T01:19:27.068698+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:33.925924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068473+0000","last_change":"2023-07-06T01:19:24.643437+0000","last_active":"2023-07-06T01:19:27.068473+0000","last_peered":"2023-07-06T01:19:27.068473+0000","last_clean":"2023-07-06T01:19:27.068473+0000","last_became_active":"2023-07-06T01:19:24.642926+0000","last_became_peered":"2023-07-06T01:19:24.642926+0000","last_unstale":"2023-07-06T01:19:27.068473+0000","last_undegraded":"2023-07-06T01:19:27.068473+0000","last_fullsized":"2023-07-06T01:19:27.068473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:17.543019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068999+0000","last_change":"2023-07-06T01:19:25.656081+0000","last_active":"2023-07-06T01:19:27.068999+0000","last_peered":"2023-07-06T01:19:27.068999+0000","last_clean":"2023-07-06T01:19:27.068999+0000","last_became_active":"2023-07-06T01:19:25.655801+0000","last_became_peered":"2023-07-06T01:19:25.655801+0000","last_unstale":"2023-07-06T01:19:27.068999+0000","last_undegraded":"2023-07-06T01:19:27.068999+0000","last_fullsized":"2023-07-06T01:19:27.068999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:10.760963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068626+0000","last_change":"2023-07-06T01:19:19.586626+0000","last_active":"2023-07-06T01:19:27.068626+0000","last_peered":"2023-07-06T01:19:27.068626+0000","last_clean":"2023-07-06T01:19:27.068626+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:19:27.068626+0000","last_undegraded":"2023-07-06T01:19:27.068626+0000","last_fullsized":"2023-07-06T01:19:27.068626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:30.708237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944310+0000","last_change":"2023-07-06T01:19:24.644389+0000","last_active":"2023-07-06T01:19:26.944310+0000","last_peered":"2023-07-06T01:19:26.944310+0000","last_clean":"2023-07-06T01:19:26.944310+0000","last_became_active":"2023-07-06T01:19:24.644113+0000","last_became_peered":"2023-07-06T01:19:24.644113+0000","last_unstale":"2023-07-06T01:19:26.944310+0000","last_undegraded":"2023-07-06T01:19:26.944310+0000","last_fullsized":"2023-07-06T01:19:26.944310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:30.566175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069023+0000","last_change":"2023-07-06T01:19:25.654127+0000","last_active":"2023-07-06T01:19:27.069023+0000","last_peered":"2023-07-06T01:19:27.069023+0000","last_clean":"2023-07-06T01:19:27.069023+0000","last_became_active":"2023-07-06T01:19:25.653903+0000","last_became_peered":"2023-07-06T01:19:25.653903+0000","last_unstale":"2023-07-06T01:19:27.069023+0000","last_undegraded":"2023-07-06T01:19:27.069023+0000","last_fullsized":"2023-07-06T01:19:27.069023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:25.693365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631434+0000","last_change":"2023-07-06T01:19:19.584086+0000","last_active":"2023-07-06T01:19:26.631434+0000","last_peered":"2023-07-06T01:19:26.631434+0000","last_clean":"2023-07-06T01:19:26.631434+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:19:26.631434+0000","last_undegraded":"2023-07-06T01:19:26.631434+0000","last_fullsized":"2023-07-06T01:19:26.631434+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:36.747665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632572+0000","last_change":"2023-07-06T01:19:24.648405+0000","last_active":"2023-07-06T01:19:26.632572+0000","last_peered":"2023-07-06T01:19:26.632572+0000","last_clean":"2023-07-06T01:19:26.632572+0000","last_became_active":"2023-07-06T01:19:24.648161+0000","last_became_peered":"2023-07-06T01:19:24.648161+0000","last_unstale":"2023-07-06T01:19:26.632572+0000","last_undegraded":"2023-07-06T01:19:26.632572+0000","last_fullsized":"2023-07-06T01:19:26.632572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:39:24.607769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069089+0000","last_change":"2023-07-06T01:19:25.641885+0000","last_active":"2023-07-06T01:19:27.069089+0000","last_peered":"2023-07-06T01:19:27.069089+0000","last_clean":"2023-07-06T01:19:27.069089+0000","last_became_active":"2023-07-06T01:19:25.641570+0000","last_became_peered":"2023-07-06T01:19:25.641570+0000","last_unstale":"2023-07-06T01:19:27.069089+0000","last_undegraded":"2023-07-06T01:19:27.069089+0000","last_fullsized":"2023-07-06T01:19:27.069089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:09.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069072+0000","last_change":"2023-07-06T01:19:19.586424+0000","last_active":"2023-07-06T01:19:27.069072+0000","last_peered":"2023-07-06T01:19:27.069072+0000","last_clean":"2023-07-06T01:19:27.069072+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:19:27.069072+0000","last_undegraded":"2023-07-06T01:19:27.069072+0000","last_fullsized":"2023-07-06T01:19:27.069072+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:15.511670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068948+0000","last_change":"2023-07-06T01:19:24.644635+0000","last_active":"2023-07-06T01:19:27.068948+0000","last_peered":"2023-07-06T01:19:27.068948+0000","last_clean":"2023-07-06T01:19:27.068948+0000","last_became_active":"2023-07-06T01:19:24.644344+0000","last_became_peered":"2023-07-06T01:19:24.644344+0000","last_unstale":"2023-07-06T01:19:27.068948+0000","last_undegraded":"2023-07-06T01:19:27.068948+0000","last_fullsized":"2023-07-06T01:19:27.068948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:39.207991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631123+0000","last_change":"2023-07-06T01:19:25.657020+0000","last_active":"2023-07-06T01:19:26.631123+0000","last_peered":"2023-07-06T01:19:26.631123+0000","last_clean":"2023-07-06T01:19:26.631123+0000","last_became_active":"2023-07-06T01:19:25.656084+0000","last_became_peered":"2023-07-06T01:19:25.656084+0000","last_unstale":"2023-07-06T01:19:26.631123+0000","last_undegraded":"2023-07-06T01:19:26.631123+0000","last_fullsized":"2023-07-06T01:19:26.631123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:22.147924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069477+0000","last_change":"2023-07-06T01:19:19.583699+0000","last_active":"2023-07-06T01:19:27.069477+0000","last_peered":"2023-07-06T01:19:27.069477+0000","last_clean":"2023-07-06T01:19:27.069477+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:19:27.069477+0000","last_undegraded":"2023-07-06T01:19:27.069477+0000","last_fullsized":"2023-07-06T01:19:27.069477+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:04.706590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631992+0000","last_change":"2023-07-06T01:19:24.642998+0000","last_active":"2023-07-06T01:19:26.631992+0000","last_peered":"2023-07-06T01:19:26.631992+0000","last_clean":"2023-07-06T01:19:26.631992+0000","last_became_active":"2023-07-06T01:19:24.642674+0000","last_became_peered":"2023-07-06T01:19:24.642674+0000","last_unstale":"2023-07-06T01:19:26.631992+0000","last_undegraded":"2023-07-06T01:19:26.631992+0000","last_fullsized":"2023-07-06T01:19:26.631992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:44.850783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068822+0000","last_change":"2023-07-06T01:19:25.651577+0000","last_active":"2023-07-06T01:19:27.068822+0000","last_peered":"2023-07-06T01:19:27.068822+0000","last_clean":"2023-07-06T01:19:27.068822+0000","last_became_active":"2023-07-06T01:19:25.651299+0000","last_became_peered":"2023-07-06T01:19:25.651299+0000","last_unstale":"2023-07-06T01:19:27.068822+0000","last_undegraded":"2023-07-06T01:19:27.068822+0000","last_fullsized":"2023-07-06T01:19:27.068822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:25.693365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631194+0000","last_change":"2023-07-06T01:19:24.648687+0000","last_active":"2023-07-06T01:19:26.631194+0000","last_peered":"2023-07-06T01:19:26.631194+0000","last_clean":"2023-07-06T01:19:26.631194+0000","last_became_active":"2023-07-06T01:19:24.648363+0000","last_became_peered":"2023-07-06T01:19:24.648363+0000","last_unstale":"2023-07-06T01:19:26.631194+0000","last_undegraded":"2023-07-06T01:19:26.631194+0000","last_fullsized":"2023-07-06T01:19:26.631194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:24.719467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069057+0000","last_change":"2023-07-06T01:19:25.646102+0000","last_active":"2023-07-06T01:19:27.069057+0000","last_peered":"2023-07-06T01:19:27.069057+0000","last_clean":"2023-07-06T01:19:27.069057+0000","last_became_active":"2023-07-06T01:19:25.645911+0000","last_became_peered":"2023-07-06T01:19:25.645911+0000","last_unstale":"2023-07-06T01:19:27.069057+0000","last_undegraded":"2023-07-06T01:19:27.069057+0000","last_fullsized":"2023-07-06T01:19:27.069057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:58:05.681285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631273+0000","last_change":"2023-07-06T01:19:24.645915+0000","last_active":"2023-07-06T01:19:26.631273+0000","last_peered":"2023-07-06T01:19:26.631273+0000","last_clean":"2023-07-06T01:19:26.631273+0000","last_became_active":"2023-07-06T01:19:24.645626+0000","last_became_peered":"2023-07-06T01:19:24.645626+0000","last_unstale":"2023-07-06T01:19:26.631273+0000","last_undegraded":"2023-07-06T01:19:26.631273+0000","last_fullsized":"2023-07-06T01:19:26.631273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:32.163145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632582+0000","last_change":"2023-07-06T01:19:25.658031+0000","last_active":"2023-07-06T01:19:26.632582+0000","last_peered":"2023-07-06T01:19:26.632582+0000","last_clean":"2023-07-06T01:19:26.632582+0000","last_became_active":"2023-07-06T01:19:25.657733+0000","last_became_peered":"2023-07-06T01:19:25.657733+0000","last_unstale":"2023-07-06T01:19:26.632582+0000","last_undegraded":"2023-07-06T01:19:26.632582+0000","last_fullsized":"2023-07-06T01:19:26.632582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:35:30.587396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632124+0000","last_change":"2023-07-06T01:19:24.645784+0000","last_active":"2023-07-06T01:19:26.632124+0000","last_peered":"2023-07-06T01:19:26.632124+0000","last_clean":"2023-07-06T01:19:26.632124+0000","last_became_active":"2023-07-06T01:19:24.645447+0000","last_became_peered":"2023-07-06T01:19:24.645447+0000","last_unstale":"2023-07-06T01:19:26.632124+0000","last_undegraded":"2023-07-06T01:19:26.632124+0000","last_fullsized":"2023-07-06T01:19:26.632124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:08.965590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632116+0000","last_change":"2023-07-06T01:19:25.653462+0000","last_active":"2023-07-06T01:19:26.632116+0000","last_peered":"2023-07-06T01:19:26.632116+0000","last_clean":"2023-07-06T01:19:26.632116+0000","last_became_active":"2023-07-06T01:19:25.652772+0000","last_became_peered":"2023-07-06T01:19:25.652772+0000","last_unstale":"2023-07-06T01:19:26.632116+0000","last_undegraded":"2023-07-06T01:19:26.632116+0000","last_fullsized":"2023-07-06T01:19:26.632116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:01:59.139466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944733+0000","last_change":"2023-07-06T01:19:24.642392+0000","last_active":"2023-07-06T01:19:26.944733+0000","last_peered":"2023-07-06T01:19:26.944733+0000","last_clean":"2023-07-06T01:19:26.944733+0000","last_became_active":"2023-07-06T01:19:24.642122+0000","last_became_peered":"2023-07-06T01:19:24.642122+0000","last_unstale":"2023-07-06T01:19:26.944733+0000","last_undegraded":"2023-07-06T01:19:26.944733+0000","last_fullsized":"2023-07-06T01:19:26.944733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:01:58.134250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944450+0000","last_change":"2023-07-06T01:19:25.650007+0000","last_active":"2023-07-06T01:19:26.944450+0000","last_peered":"2023-07-06T01:19:26.944450+0000","last_clean":"2023-07-06T01:19:26.944450+0000","last_became_active":"2023-07-06T01:19:25.649108+0000","last_became_peered":"2023-07-06T01:19:25.649108+0000","last_unstale":"2023-07-06T01:19:26.944450+0000","last_undegraded":"2023-07-06T01:19:26.944450+0000","last_fullsized":"2023-07-06T01:19:26.944450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:22.147924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630751+0000","last_change":"2023-07-06T01:19:24.642694+0000","last_active":"2023-07-06T01:19:26.630751+0000","last_peered":"2023-07-06T01:19:26.630751+0000","last_clean":"2023-07-06T01:19:26.630751+0000","last_became_active":"2023-07-06T01:19:24.642130+0000","last_became_peered":"2023-07-06T01:19:24.642130+0000","last_unstale":"2023-07-06T01:19:26.630751+0000","last_undegraded":"2023-07-06T01:19:26.630751+0000","last_fullsized":"2023-07-06T01:19:26.630751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:09.755748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631214+0000","last_change":"2023-07-06T01:19:25.653965+0000","last_active":"2023-07-06T01:19:26.631214+0000","last_peered":"2023-07-06T01:19:26.631214+0000","last_clean":"2023-07-06T01:19:26.631214+0000","last_became_active":"2023-07-06T01:19:25.653448+0000","last_became_peered":"2023-07-06T01:19:25.653448+0000","last_unstale":"2023-07-06T01:19:26.631214+0000","last_undegraded":"2023-07-06T01:19:26.631214+0000","last_fullsized":"2023-07-06T01:19:26.631214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:09.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068713+0000","last_change":"2023-07-06T01:19:25.652413+0000","last_active":"2023-07-06T01:19:27.068713+0000","last_peered":"2023-07-06T01:19:27.068713+0000","last_clean":"2023-07-06T01:19:27.068713+0000","last_became_active":"2023-07-06T01:19:25.652051+0000","last_became_peered":"2023-07-06T01:19:25.652051+0000","last_unstale":"2023-07-06T01:19:27.068713+0000","last_undegraded":"2023-07-06T01:19:27.068713+0000","last_fullsized":"2023-07-06T01:19:27.068713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:44.291645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944695+0000","last_change":"2023-07-06T01:19:25.652994+0000","last_active":"2023-07-06T01:19:26.944695+0000","last_peered":"2023-07-06T01:19:26.944695+0000","last_clean":"2023-07-06T01:19:26.944695+0000","last_became_active":"2023-07-06T01:19:25.652484+0000","last_became_peered":"2023-07-06T01:19:25.652484+0000","last_unstale":"2023-07-06T01:19:26.944695+0000","last_undegraded":"2023-07-06T01:19:26.944695+0000","last_fullsized":"2023-07-06T01:19:26.944695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:14.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631822+0000","last_change":"2023-07-06T01:19:25.656779+0000","last_active":"2023-07-06T01:19:26.631822+0000","last_peered":"2023-07-06T01:19:26.631822+0000","last_clean":"2023-07-06T01:19:26.631822+0000","last_became_active":"2023-07-06T01:19:25.653104+0000","last_became_peered":"2023-07-06T01:19:25.653104+0000","last_unstale":"2023-07-06T01:19:26.631822+0000","last_undegraded":"2023-07-06T01:19:26.631822+0000","last_fullsized":"2023-07-06T01:19:26.631822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:41:34.533716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944583+0000","last_change":"2023-07-06T01:19:25.653857+0000","last_active":"2023-07-06T01:19:26.944583+0000","last_peered":"2023-07-06T01:19:26.944583+0000","last_clean":"2023-07-06T01:19:26.944583+0000","last_became_active":"2023-07-06T01:19:25.652976+0000","last_became_peered":"2023-07-06T01:19:25.652976+0000","last_unstale":"2023-07-06T01:19:26.944583+0000","last_undegraded":"2023-07-06T01:19:26.944583+0000","last_fullsized":"2023-07-06T01:19:26.944583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:25.724682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068930+0000","last_change":"2023-07-06T01:19:25.647146+0000","last_active":"2023-07-06T01:19:27.068930+0000","last_peered":"2023-07-06T01:19:27.068930+0000","last_clean":"2023-07-06T01:19:27.068930+0000","last_became_active":"2023-07-06T01:19:25.646863+0000","last_became_peered":"2023-07-06T01:19:25.646863+0000","last_unstale":"2023-07-06T01:19:27.068930+0000","last_undegraded":"2023-07-06T01:19:27.068930+0000","last_fullsized":"2023-07-06T01:19:27.068930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:21.566043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":670687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":233472,"data_stored":80159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":794624,"data_stored":662254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":78539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.248}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":78629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":79191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":200704,"data_stored":71886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:19:30.190 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T01:19:30.190 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-06T01:19:30.191 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-07-06T01:19:30.191 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-07-06T01:19:30.192 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-07-06T01:19:30.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.342+0000 7fbf28ef9700 1 Processor -- start 2023-07-06T01:19:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.343+0000 7fbf28ef9700 1 -- start start 2023-07-06T01:19:30.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.343+0000 7fbf28ef9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbf2415a610 0x7fbf2406b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:30.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.343+0000 7fbf28ef9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fbf2415eae0 0x7fbf2406c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:30.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.343+0000 7fbf28ef9700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf240caa50 con 0x7fbf2406c730 2023-07-06T01:19:30.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.343+0000 7fbf28ef9700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2415a9e0 con 0x7fbf2415a610 2023-07-06T01:19:30.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.343+0000 7fbf28ef9700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2406cbb0 con 0x7fbf2415eae0 2023-07-06T01:19:30.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.344+0000 7fbf2259c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbf2415a610 0x7fbf2406b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:30.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.344+0000 7fbf2259c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbf2415a610 0x7fbf2406b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45352/0 (socket says 172.21.15.22:45352) 2023-07-06T01:19:30.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.344+0000 7fbf2259c700 1 -- 172.21.15.22:0/4193611487 learned_addr learned my addr 172.21.15.22:0/4193611487 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:30.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.344+0000 7fbf21d9b700 1 --2- 172.21.15.22:0/4193611487 >> v2:172.21.15.103:3300/0 conn(0x7fbf2415eae0 0x7fbf2406c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:30.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.344+0000 7fbf2259c700 1 -- 172.21.15.22:0/4193611487 >> v2:172.21.15.103:3300/0 conn(0x7fbf2415eae0 msgr2=0x7fbf2406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:30.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.344+0000 7fbf2259c700 1 --2- 172.21.15.22:0/4193611487 >> v2:172.21.15.103:3300/0 conn(0x7fbf2415eae0 0x7fbf2406c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.344+0000 7fbf2259c700 1 -- 172.21.15.22:0/4193611487 >> v1:172.21.15.103:6789/0 conn(0x7fbf2406c730 legacy=0x7fbf2406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.344+0000 7fbf2259c700 1 -- 172.21.15.22:0/4193611487 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf24071f30 con 0x7fbf2415a610 2023-07-06T01:19:30.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.344+0000 7fbf2259c700 1 --2- 172.21.15.22:0/4193611487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf2415a610 0x7fbf2406b320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbf240ae810 tx=0x7fbf14009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a15c773af9b33d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:30.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.344+0000 7fbf2159a700 1 -- 172.21.15.22:0/4193611487 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400b870 con 0x7fbf2415a610 2023-07-06T01:19:30.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.344+0000 7fbf2159a700 1 -- 172.21.15.22:0/4193611487 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1400b9d0 con 0x7fbf2415a610 2023-07-06T01:19:30.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.344+0000 7fbf2159a700 1 -- 172.21.15.22:0/4193611487 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400bd60 con 0x7fbf2415a610 2023-07-06T01:19:30.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.345+0000 7fbf28ef9700 1 -- 172.21.15.22:0/4193611487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf2415a610 msgr2=0x7fbf2406b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.345+0000 7fbf28ef9700 1 --2- 172.21.15.22:0/4193611487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf2415a610 0x7fbf2406b320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbf240ae810 tx=0x7fbf14009bd0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.345+0000 7fbf28ef9700 1 -- 172.21.15.22:0/4193611487 shutdown_connections 2023-07-06T01:19:30.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.345+0000 7fbf28ef9700 1 --2- 172.21.15.22:0/4193611487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf2415a610 0x7fbf2406b320 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.345+0000 7fbf28ef9700 1 --2- 172.21.15.22:0/4193611487 >> v2:172.21.15.103:3300/0 conn(0x7fbf2415eae0 0x7fbf2406c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.345+0000 7fbf28ef9700 1 -- 172.21.15.22:0/4193611487 >> 172.21.15.22:0/4193611487 conn(0x7fbf240ccb00 msgr2=0x7fbf240cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:30.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.345+0000 7fbf28ef9700 1 -- 172.21.15.22:0/4193611487 shutdown_connections 2023-07-06T01:19:30.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.345+0000 7fbf28ef9700 1 -- 172.21.15.22:0/4193611487 wait complete. 2023-07-06T01:19:30.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.346+0000 7fbf28ef9700 1 Processor -- start 2023-07-06T01:19:30.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.346+0000 7fbf28ef9700 1 -- start start 2023-07-06T01:19:30.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf28ef9700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf2406c730 0x7fbf24154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:30.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf28ef9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf2415a610 0x7fbf240c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:30.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf28ef9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf2415eae0 0x7fbf2415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:30.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf28ef9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf241e5a30 con 0x7fbf2415eae0 2023-07-06T01:19:30.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf28ef9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf241e4990 con 0x7fbf2415a610 2023-07-06T01:19:30.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf28ef9700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf241e4c70 con 0x7fbf2406c730 2023-07-06T01:19:30.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf22d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf2415eae0 0x7fbf2415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:30.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf22d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf2415eae0 0x7fbf2415cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45360/0 (socket says 172.21.15.22:45360) 2023-07-06T01:19:30.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf22d9d700 1 -- 172.21.15.22:0/3186369360 learned_addr learned my addr 172.21.15.22:0/3186369360 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:30.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf21d9b700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf2415a610 0x7fbf240c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:30.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf2259c700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf2406c730 0x7fbf24154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:30.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf22d9d700 1 -- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf2406c730 msgr2=0x7fbf24154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf22d9d700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf2406c730 0x7fbf24154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf22d9d700 1 -- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf2415a610 msgr2=0x7fbf240c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf22d9d700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf2415a610 0x7fbf240c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf22d9d700 1 -- 172.21.15.22:0/3186369360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf14009550 con 0x7fbf2415eae0 2023-07-06T01:19:30.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.347+0000 7fbf21d9b700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf2415a610 0x7fbf240c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:30.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.348+0000 7fbf22d9d700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf2415eae0 0x7fbf2415cfd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00cc50 tx=0x7fbf1c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:30.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.348+0000 7fbf1affd700 1 -- 172.21.15.22:0/3186369360 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c00ad80 con 0x7fbf2415eae0 2023-07-06T01:19:30.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.348+0000 7fbf1affd700 1 -- 172.21.15.22:0/3186369360 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1c00ce50 con 0x7fbf2415eae0 2023-07-06T01:19:30.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.348+0000 7fbf28ef9700 1 -- 172.21.15.22:0/3186369360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf241e4f50 con 0x7fbf2415eae0 2023-07-06T01:19:30.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.348+0000 7fbf1affd700 1 -- 172.21.15.22:0/3186369360 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c003c40 con 0x7fbf2415eae0 2023-07-06T01:19:30.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.348+0000 7fbf28ef9700 1 -- 172.21.15.22:0/3186369360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf241e5510 con 0x7fbf2415eae0 2023-07-06T01:19:30.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.349+0000 7fbf28ef9700 1 -- 172.21.15.22:0/3186369360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf241e5730 con 0x7fbf2415eae0 2023-07-06T01:19:30.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.349+0000 7fbf1affd700 1 -- 172.21.15.22:0/3186369360 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf1c0050d0 con 0x7fbf2415eae0 2023-07-06T01:19:30.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.350+0000 7fbf1affd700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbf0406ace0 0x7fbf0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:30.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.350+0000 7fbf1affd700 1 -- 172.21.15.22:0/3186369360 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf1c089590 con 0x7fbf2415eae0 2023-07-06T01:19:30.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.350+0000 7fbf2259c700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbf0406ace0 0x7fbf0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:30.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.350+0000 7fbf2259c700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbf0406ace0 0x7fbf0406d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbf241f3750 tx=0x7fbf14008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:30.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.354+0000 7fbf1affd700 1 -- 172.21.15.22:0/3186369360 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbf1c054640 con 0x7fbf2415eae0 2023-07-06T01:19:30.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.609+0000 7fbf28ef9700 1 -- 172.21.15.22:0/3186369360 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fbf240744b0 con 0x7fbf2415eae0 2023-07-06T01:19:30.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.612+0000 7fbf1affd700 1 -- 172.21.15.22:0/3186369360 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170846 (secure 0 0 0) 0x7fbf1c057c60 con 0x7fbf2415eae0 2023-07-06T01:19:30.616 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:30.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.615+0000 7fbf28ef9700 1 -- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbf0406ace0 msgr2=0x7fbf0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.615+0000 7fbf28ef9700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbf0406ace0 0x7fbf0406d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbf241f3750 tx=0x7fbf14008030 comp rx=0 tx=0).stop 2023-07-06T01:19:30.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.615+0000 7fbf28ef9700 1 -- 172.21.15.22:0/3186369360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf2415eae0 msgr2=0x7fbf2415cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.615+0000 7fbf28ef9700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf2415eae0 0x7fbf2415cfd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00cc50 tx=0x7fbf1c00a340 comp rx=0 tx=0).stop 2023-07-06T01:19:30.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.615+0000 7fbf28ef9700 1 -- 172.21.15.22:0/3186369360 shutdown_connections 2023-07-06T01:19:30.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.615+0000 7fbf28ef9700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf2415eae0 0x7fbf2415cfd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.615+0000 7fbf28ef9700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbf2406c730 0x7fbf24154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.615+0000 7fbf28ef9700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbf0406ace0 0x7fbf0406d190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.615+0000 7fbf28ef9700 1 --2- 172.21.15.22:0/3186369360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbf2415a610 0x7fbf240c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.615+0000 7fbf28ef9700 1 -- 172.21.15.22:0/3186369360 >> 172.21.15.22:0/3186369360 conn(0x7fbf240ccb00 msgr2=0x7fbf240afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:30.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.615+0000 7fbf28ef9700 1 -- 172.21.15.22:0/3186369360 shutdown_connections 2023-07-06T01:19:30.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.615+0000 7fbf28ef9700 1 -- 172.21.15.22:0/3186369360 wait complete. 2023-07-06T01:19:30.628 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":5,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1225712448},{"type":"v1","addr":"172.21.15.103:6833","nonce":1225712448}]},"active_addr":"172.21.15.103:6833/1225712448","active_change":"2023-07-06T01:19:11.990185+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":2621497303}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":4088718}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":2464121172}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":1182302653}]}]} 2023-07-06T01:19:30.634 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-07-06T01:19:30.635 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-07-06T01:19:30.635 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:30.712 INFO:tasks.ceph.mds.d.smithi103.stderr:2023-07-06T01:19:30.709+0000 7f76b51ea700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T01:19:30.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.759+0000 7f83b2324700 1 Processor -- start 2023-07-06T01:19:30.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.759+0000 7f83b2324700 1 -- start start 2023-07-06T01:19:30.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83b2324700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f83ac0af2a0 0x7f83ac06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:30.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83b2324700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83ac06a000 0x7f83ac06a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:30.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83b2324700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ac0af670 con 0x7f83ac0cbc10 2023-07-06T01:19:30.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83b2324700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ac06a9e0 con 0x7f83ac0af2a0 2023-07-06T01:19:30.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83b2324700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac0cc7a0 con 0x7f83ac06a000 2023-07-06T01:19:30.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83ab7fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83ac06a000 0x7f83ac06a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:30.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83abfff700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f83ac0af2a0 0x7f83ac06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:30.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83ab7fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83ac06a000 0x7f83ac06a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56506/0 (socket says 172.21.15.22:56506) 2023-07-06T01:19:30.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83abfff700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f83ac0af2a0 0x7f83ac06caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:34090/0 (socket says 172.21.15.22:34090) 2023-07-06T01:19:30.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83b08c1700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f83ac0cbc10 0x7f83ac0cc090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38178/0 (socket says 172.21.15.22:38178) 2023-07-06T01:19:30.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83ab7fe700 1 -- 172.21.15.22:0/4063290290 learned_addr learned my addr 172.21.15.22:0/4063290290 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:30.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83aaffd700 1 -- 172.21.15.22:0/4063290290 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348102410 0 0) 0x7f83ac0af670 con 0x7f83ac0cbc10 2023-07-06T01:19:30.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83aaffd700 1 -- 172.21.15.22:0/4063290290 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83940035b0 con 0x7f83ac0cbc10 2023-07-06T01:19:30.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.760+0000 7f83abfff700 1 -- 172.21.15.22:0/4063290290 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83ac06a000 msgr2=0x7f83ac06a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:30.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83abfff700 1 --2- 172.21.15.22:0/4063290290 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83ac06a000 0x7f83ac06a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83abfff700 1 -- 172.21.15.22:0/4063290290 >> v1:172.21.15.103:6789/0 conn(0x7f83ac0cbc10 legacy=0x7f83ac0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83abfff700 1 -- 172.21.15.22:0/4063290290 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac06d030 con 0x7f83ac0af2a0 2023-07-06T01:19:30.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83ab7fe700 1 --2- 172.21.15.22:0/4063290290 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83ac06a000 0x7f83ac06a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:30.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83abfff700 1 --2- 172.21.15.22:0/4063290290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83ac0af2a0 0x7f83ac06caf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f839c009840 tx=0x7f839c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a1d8c84cb7420e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:30.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83aaffd700 1 -- 172.21.15.22:0/4063290290 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c011830 con 0x7f83ac0af2a0 2023-07-06T01:19:30.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83aaffd700 1 -- 172.21.15.22:0/4063290290 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c011990 con 0x7f83ac0af2a0 2023-07-06T01:19:30.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83aaffd700 1 -- 172.21.15.22:0/4063290290 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c011d20 con 0x7f83ac0af2a0 2023-07-06T01:19:30.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83b2324700 1 -- 172.21.15.22:0/4063290290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83ac0af2a0 msgr2=0x7f83ac06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83b2324700 1 --2- 172.21.15.22:0/4063290290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83ac0af2a0 0x7f83ac06caf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f839c009840 tx=0x7f839c009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:30.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83b2324700 1 -- 172.21.15.22:0/4063290290 shutdown_connections 2023-07-06T01:19:30.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83b2324700 1 --2- 172.21.15.22:0/4063290290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83ac0af2a0 0x7f83ac06caf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83b2324700 1 --2- 172.21.15.22:0/4063290290 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83ac06a000 0x7f83ac06a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83b2324700 1 -- 172.21.15.22:0/4063290290 >> 172.21.15.22:0/4063290290 conn(0x7f83ac0b1890 msgr2=0x7f83ac0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:30.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.761+0000 7f83b2324700 1 -- 172.21.15.22:0/4063290290 shutdown_connections 2023-07-06T01:19:30.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.762+0000 7f83b2324700 1 -- 172.21.15.22:0/4063290290 wait complete. 2023-07-06T01:19:30.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.763+0000 7f83b2324700 1 Processor -- start 2023-07-06T01:19:30.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.763+0000 7f83b2324700 1 -- start start 2023-07-06T01:19:30.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b2324700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83ac06a000 0x7f83ac1f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:30.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b2324700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83ac0af2a0 0x7f83ac0b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:30.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b2324700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83ac0cbc10 0x7f83ac0b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:30.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b2324700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac0b59f0 con 0x7f83ac0cbc10 2023-07-06T01:19:30.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b2324700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac1e5280 con 0x7f83ac06a000 2023-07-06T01:19:30.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b2324700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac0b3840 con 0x7f83ac0af2a0 2023-07-06T01:19:30.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b08c1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83ac0cbc10 0x7f83ac0b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:30.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b08c1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83ac0cbc10 0x7f83ac0b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45376/0 (socket says 172.21.15.22:45376) 2023-07-06T01:19:30.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b08c1700 1 -- 172.21.15.22:0/935747658 learned_addr learned my addr 172.21.15.22:0/935747658 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:30.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83abfff700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83ac06a000 0x7f83ac1f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:30.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83ab7fe700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83ac0af2a0 0x7f83ac0b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:30.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b08c1700 1 -- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83ac0af2a0 msgr2=0x7f83ac0b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b08c1700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83ac0af2a0 0x7f83ac0b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b08c1700 1 -- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83ac06a000 msgr2=0x7f83ac1f5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b08c1700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83ac06a000 0x7f83ac1f5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b08c1700 1 -- 172.21.15.22:0/935747658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0009550 con 0x7f83ac0cbc10 2023-07-06T01:19:30.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b08c1700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83ac0cbc10 0x7f83ac0b54b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f83a400cbc0 tx=0x7f83a400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:30.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83a8ff9700 1 -- 172.21.15.22:0/935747658 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a400aa40 con 0x7f83ac0cbc10 2023-07-06T01:19:30.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83a8ff9700 1 -- 172.21.15.22:0/935747658 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a400aba0 con 0x7f83ac0cbc10 2023-07-06T01:19:30.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.764+0000 7f83b2324700 1 -- 172.21.15.22:0/935747658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839c0094a0 con 0x7f83ac0cbc10 2023-07-06T01:19:30.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.765+0000 7f83a8ff9700 1 -- 172.21.15.22:0/935747658 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4007070 con 0x7f83ac0cbc10 2023-07-06T01:19:30.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.765+0000 7f83b2324700 1 -- 172.21.15.22:0/935747658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ac1e54d0 con 0x7f83ac0cbc10 2023-07-06T01:19:30.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.766+0000 7f83b2324700 1 -- 172.21.15.22:0/935747658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ac0744b0 con 0x7f83ac0cbc10 2023-07-06T01:19:30.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.766+0000 7f83a8ff9700 1 -- 172.21.15.22:0/935747658 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f83a40071d0 con 0x7f83ac0cbc10 2023-07-06T01:19:30.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.766+0000 7f83a8ff9700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f839806ace0 0x7f839806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:30.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.766+0000 7f83a8ff9700 1 -- 172.21.15.22:0/935747658 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f83a4089160 con 0x7f83ac0cbc10 2023-07-06T01:19:30.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.767+0000 7f83abfff700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f839806ace0 0x7f839806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:30.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.767+0000 7f83abfff700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f839806ace0 0x7f839806d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f839c000c00 tx=0x7f839c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:30.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.771+0000 7f83a8ff9700 1 -- 172.21.15.22:0/935747658 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f83a40542a0 con 0x7f83ac0cbc10 2023-07-06T01:19:30.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.970+0000 7f83b2324700 1 -- 172.21.15.22:0/935747658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83ac15ca90 con 0x7f83ac0cbc10 2023-07-06T01:19:30.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.971+0000 7f83a8ff9700 1 -- 172.21.15.22:0/935747658 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7f83ac15ca90 con 0x7f83ac0cbc10 2023-07-06T01:19:30.978 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:30.978 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:30.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.973+0000 7f83b2324700 1 -- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f839806ace0 msgr2=0x7f839806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.973+0000 7f83b2324700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f839806ace0 0x7f839806d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f839c000c00 tx=0x7f839c008030 comp rx=0 tx=0).stop 2023-07-06T01:19:30.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.973+0000 7f83b2324700 1 -- 172.21.15.22:0/935747658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83ac0cbc10 msgr2=0x7f83ac0b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:30.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.973+0000 7f83b2324700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83ac0cbc10 0x7f83ac0b54b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f83a400cbc0 tx=0x7f83a400ced0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.973+0000 7f83b2324700 1 -- 172.21.15.22:0/935747658 shutdown_connections 2023-07-06T01:19:30.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.973+0000 7f83b2324700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83ac0cbc10 0x7f83ac0b54b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.973+0000 7f83b2324700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f839806ace0 0x7f839806d190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.973+0000 7f83b2324700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83ac06a000 0x7f83ac1f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.973+0000 7f83b2324700 1 --2- 172.21.15.22:0/935747658 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83ac0af2a0 0x7f83ac0b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:30.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.973+0000 7f83b2324700 1 -- 172.21.15.22:0/935747658 >> 172.21.15.22:0/935747658 conn(0x7f83ac0b1890 msgr2=0x7f83ac0cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:30.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.973+0000 7f83b2324700 1 -- 172.21.15.22:0/935747658 shutdown_connections 2023-07-06T01:19:30.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:30.973+0000 7f83b2324700 1 -- 172.21.15.22:0/935747658 wait complete. 2023-07-06T01:19:30.987 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-07-06T01:19:30.988 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:31.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.111+0000 7f9515409700 1 Processor -- start 2023-07-06T01:19:31.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.111+0000 7f9515409700 1 -- start start 2023-07-06T01:19:31.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.111+0000 7f9515409700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951015d9e0 0x7f951006d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.111+0000 7f9515409700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95100cf230 con 0x7f95100ccd70 2023-07-06T01:19:31.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.111+0000 7f9515409700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95100ce620 con 0x7f9510159360 2023-07-06T01:19:31.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.111+0000 7f9515409700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95100ce900 con 0x7f951015d9e0 2023-07-06T01:19:31.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.111+0000 7f950e7fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f95100ccd70 0x7f951006d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36124/0 (socket says 172.21.15.22:36124) 2023-07-06T01:19:31.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950e7fc700 1 -- 172.21.15.22:0/994830 learned_addr learned my addr 172.21.15.22:0/994830 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:31.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767158525 0 0) 0x7f95100cf230 con 0x7f95100ccd70 2023-07-06T01:19:31.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950f7fe700 1 --2- 172.21.15.22:0/994830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951015d9e0 0x7f951006d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94f80035b0 con 0x7f95100ccd70 2023-07-06T01:19:31.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1009809469 0 0) 0x7f94f80035b0 con 0x7f95100ccd70 2023-07-06T01:19:31.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951015d9e0 msgr2=0x7f951006d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 --2- 172.21.15.22:0/994830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951015d9e0 0x7f951006d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 >> v1:172.21.15.103:6789/0 conn(0x7f9510159360 legacy=0x7f95100cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95100cebe0 con 0x7f95100ccd70 2023-07-06T01:19:31.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f9500001670 con 0x7f95100ccd70 2023-07-06T01:19:31.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 >> v1:172.21.15.22:6789/0 conn(0x7f95100ccd70 legacy=0x7f951006d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 --2- 172.21.15.22:0/994830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951015d9e0 0x7f94f8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 --2- 172.21.15.22:0/994830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f8003fb0 0x7f94f8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 --2- 172.21.15.22:0/994830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94f8004d90 0x7f94f8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95100ce900 con 0x7f94f8003fb0 2023-07-06T01:19:31.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f800baf0 con 0x7f94f8004d90 2023-07-06T01:19:31.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f800dec0 con 0x7f951015d9e0 2023-07-06T01:19:31.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950e7fc700 1 --2- 172.21.15.22:0/994830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f8003fb0 0x7f94f8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950f7fe700 1 --2- 172.21.15.22:0/994830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94f8004d90 0x7f94f8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950effd700 1 --2- 172.21.15.22:0/994830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951015d9e0 0x7f94f8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950e7fc700 1 -- 172.21.15.22:0/994830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951015d9e0 msgr2=0x7f94f8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950e7fc700 1 --2- 172.21.15.22:0/994830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951015d9e0 0x7f94f8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950e7fc700 1 -- 172.21.15.22:0/994830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94f8004d90 msgr2=0x7f94f8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.112+0000 7f950e7fc700 1 --2- 172.21.15.22:0/994830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94f8004d90 0x7f94f8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.113+0000 7f950e7fc700 1 -- 172.21.15.22:0/994830 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f80035b0 con 0x7f94f8003fb0 2023-07-06T01:19:31.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.113+0000 7f950e7fc700 1 --2- 172.21.15.22:0/994830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f8003fb0 0x7f94f8004850 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9500001eb0 tx=0x7f94f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c71291f60c2c97f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.113+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950000cd50 con 0x7f94f8003fb0 2023-07-06T01:19:31.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.113+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9500005960 con 0x7f94f8003fb0 2023-07-06T01:19:31.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.113+0000 7f950dffb700 1 -- 172.21.15.22:0/994830 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500005d10 con 0x7f94f8003fb0 2023-07-06T01:19:31.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.113+0000 7f9515409700 1 -- 172.21.15.22:0/994830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f8003fb0 msgr2=0x7f94f8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.113+0000 7f9515409700 1 --2- 172.21.15.22:0/994830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94f8003fb0 0x7f94f8004850 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9500001eb0 tx=0x7f94f8005880 comp rx=0 tx=0).stop 2023-07-06T01:19:31.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.113+0000 7f950f7fe700 1 -- 172.21.15.22:0/994830 reap_dead start 2023-07-06T01:19:31.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.113+0000 7f9515409700 1 -- 172.21.15.22:0/994830 shutdown_connections 2023-07-06T01:19:31.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.113+0000 7f9515409700 1 -- 172.21.15.22:0/994830 >> 172.21.15.22:0/994830 conn(0x7f95100b1890 msgr2=0x7f95100b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.113+0000 7f9515409700 1 -- 172.21.15.22:0/994830 shutdown_connections 2023-07-06T01:19:31.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.113+0000 7f9515409700 1 -- 172.21.15.22:0/994830 wait complete. 2023-07-06T01:19:31.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.115+0000 7f9515409700 1 Processor -- start 2023-07-06T01:19:31.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.115+0000 7f9515409700 1 -- start start 2023-07-06T01:19:31.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.115+0000 7f9515409700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94f8003fb0 0x7f95101f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.115+0000 7f9515409700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95100b53f0 0x7f95100c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.115+0000 7f9515409700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95101540c0 0x7f9510154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.115+0000 7f9515409700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95101f2c90 con 0x7f95100b53f0 2023-07-06T01:19:31.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.115+0000 7f9515409700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9510154a70 con 0x7f95101540c0 2023-07-06T01:19:31.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.115+0000 7f9515409700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95100c77a0 con 0x7f94f8003fb0 2023-07-06T01:19:31.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f950e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95100b53f0 0x7f95100c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f950e7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95100b53f0 0x7f95100c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45394/0 (socket says 172.21.15.22:45394) 2023-07-06T01:19:31.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f950e7fc700 1 -- 172.21.15.22:0/2506024438 learned_addr learned my addr 172.21.15.22:0/2506024438 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f950effd700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94f8003fb0 0x7f95101f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f950f7fe700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95101540c0 0x7f9510154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f950e7fc700 1 -- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94f8003fb0 msgr2=0x7f95101f2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f950e7fc700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94f8003fb0 0x7f95101f2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f950e7fc700 1 -- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95101540c0 msgr2=0x7f9510154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f950e7fc700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95101540c0 0x7f9510154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f950e7fc700 1 -- 172.21.15.22:0/2506024438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95100c7d60 con 0x7f95100b53f0 2023-07-06T01:19:31.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f950e7fc700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95100b53f0 0x7f95100c81f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f95100cd1b0 tx=0x7f9500000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f94f77fe700 1 -- 172.21.15.22:0/2506024438 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500005ce0 con 0x7f95100b53f0 2023-07-06T01:19:31.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f94f77fe700 1 -- 172.21.15.22:0/2506024438 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9500005e40 con 0x7f95100b53f0 2023-07-06T01:19:31.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f94f77fe700 1 -- 172.21.15.22:0/2506024438 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500007310 con 0x7f95100b53f0 2023-07-06T01:19:31.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f9515409700 1 -- 172.21.15.22:0/2506024438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951015c120 con 0x7f95100b53f0 2023-07-06T01:19:31.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.116+0000 7f9515409700 1 -- 172.21.15.22:0/2506024438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951015c6e0 con 0x7f95100b53f0 2023-07-06T01:19:31.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.117+0000 7f9515409700 1 -- 172.21.15.22:0/2506024438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95100c7ef0 con 0x7f95100b53f0 2023-07-06T01:19:31.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.118+0000 7f94f77fe700 1 -- 172.21.15.22:0/2506024438 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9500019030 con 0x7f95100b53f0 2023-07-06T01:19:31.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.118+0000 7f94f77fe700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f94fc06ace0 0x7f94fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.118+0000 7f94f77fe700 1 -- 172.21.15.22:0/2506024438 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f950008a460 con 0x7f95100b53f0 2023-07-06T01:19:31.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.118+0000 7f950effd700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f94fc06ace0 0x7f94fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.119+0000 7f950effd700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f94fc06ace0 0x7f94fc06d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9504000c00 tx=0x7f9504009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.122+0000 7f94f77fe700 1 -- 172.21.15.22:0/2506024438 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9500055650 con 0x7f95100b53f0 2023-07-06T01:19:31.323 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.321+0000 7f9515409700 1 -- 172.21.15.22:0/2506024438 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95100744b0 con 0x7f95100b53f0 2023-07-06T01:19:31.324 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.322+0000 7f94f77fe700 1 -- 172.21.15.22:0/2506024438 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7f9500058c70 con 0x7f95100b53f0 2023-07-06T01:19:31.324 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:31.324 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.324+0000 7f9515409700 1 -- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f94fc06ace0 msgr2=0x7f94fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.324+0000 7f9515409700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f94fc06ace0 0x7f94fc06d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9504000c00 tx=0x7f9504009040 comp rx=0 tx=0).stop 2023-07-06T01:19:31.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.324+0000 7f9515409700 1 -- 172.21.15.22:0/2506024438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95100b53f0 msgr2=0x7f95100c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.324+0000 7f9515409700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95100b53f0 0x7f95100c81f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f95100cd1b0 tx=0x7f9500000fe0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.324+0000 7f9515409700 1 -- 172.21.15.22:0/2506024438 shutdown_connections 2023-07-06T01:19:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.324+0000 7f9515409700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f94fc06ace0 0x7f94fc06d190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.324+0000 7f9515409700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95101540c0 0x7f9510154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.324+0000 7f9515409700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94f8003fb0 0x7f95101f2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.324+0000 7f9515409700 1 --2- 172.21.15.22:0/2506024438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95100b53f0 0x7f95100c81f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.324+0000 7f9515409700 1 -- 172.21.15.22:0/2506024438 >> 172.21.15.22:0/2506024438 conn(0x7f95100b1890 msgr2=0x7f951015cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.324+0000 7f9515409700 1 -- 172.21.15.22:0/2506024438 shutdown_connections 2023-07-06T01:19:31.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.324+0000 7f9515409700 1 -- 172.21.15.22:0/2506024438 wait complete. 2023-07-06T01:19:31.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-07-06T01:19:31.340 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-07-06T01:19:31.340 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-07-06T01:19:31.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-07-06T01:19:31.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-07-06T01:19:31.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-07-06T01:19:31.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-07-06T01:19:31.343 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-07-06T01:19:31.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.560+0000 7f35f0099700 1 Processor -- start 2023-07-06T01:19:31.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.561+0000 7f35f0099700 1 -- start start 2023-07-06T01:19:31.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.562+0000 7f35f0099700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f35e8161010 0x7f35e806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.562+0000 7f35f0099700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35e80c2290 0x7f35e80ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.562+0000 7f35f0099700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e80cc880 con 0x7f35e80cd250 2023-07-06T01:19:31.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.562+0000 7f35f0099700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e8071a00 con 0x7f35e8161010 2023-07-06T01:19:31.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.562+0000 7f35f0099700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e806cd40 con 0x7f35e80c2290 2023-07-06T01:19:31.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.562+0000 7f35ee636700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f35e80cd250 0x7f35e806b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38212/0 (socket says 172.21.15.22:38212) 2023-07-06T01:19:31.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.562+0000 7f35ee636700 1 -- 172.21.15.22:0/2429568530 learned_addr learned my addr 172.21.15.22:0/2429568530 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:31.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.562+0000 7f35ede35700 1 --2- 172.21.15.22:0/2429568530 >> v2:172.21.15.103:3300/0 conn(0x7f35e8161010 0x7f35e806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.563+0000 7fade84a8700 1 Processor -- start 2023-07-06T01:19:31.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.563+0000 7f35ed634700 1 --2- 172.21.15.22:0/2429568530 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35e80c2290 0x7f35e80ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.563+0000 7fade84a8700 1 -- start start 2023-07-06T01:19:31.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7f35ede35700 1 -- 172.21.15.22:0/2429568530 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35e80c2290 msgr2=0x7f35e80ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7f35ede35700 1 --2- 172.21.15.22:0/2429568530 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35e80c2290 0x7f35e80ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7f35ede35700 1 -- 172.21.15.22:0/2429568530 >> v1:172.21.15.103:6789/0 conn(0x7f35e80cd250 legacy=0x7f35e806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7f35ece33700 1 -- 172.21.15.22:0/2429568530 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726342425 0 0) 0x7f35e80cc880 con 0x7f35e80cd250 2023-07-06T01:19:31.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7f35ede35700 1 -- 172.21.15.22:0/2429568530 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e806d020 con 0x7f35e8161010 2023-07-06T01:19:31.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7f35ed634700 1 --2- 172.21.15.22:0/2429568530 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35e80c2290 0x7f35e80ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:31.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7fade84a8700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fade006d3a0 0x7fade00adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7fade84a8700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fade006d770 0x7fade00ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7fade84a8700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade01613a0 con 0x7fade00acdb0 2023-07-06T01:19:31.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7fade84a8700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade00ae060 con 0x7fade006d770 2023-07-06T01:19:31.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7fade84a8700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade00cf450 con 0x7fade006d3a0 2023-07-06T01:19:31.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7f35ede35700 1 --2- 172.21.15.22:0/2429568530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35e8161010 0x7f35e806f910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009840 tx=0x7f35e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c0f3c5628497c01 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7fade5a43700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fade006d770 0x7fade00ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7fade5a43700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fade006d770 0x7fade00ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:34138/0 (socket says 172.21.15.22:34138) 2023-07-06T01:19:31.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7fade5a43700 1 -- 172.21.15.22:0/3403493750 learned_addr learned my addr 172.21.15.22:0/3403493750 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7fade6a45700 1 --1- 172.21.15.22:0/3403493750 >> v1:172.21.15.22:6789/0 conn(0x7fade00acdb0 0x7fade006b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36160/0 (socket says 172.21.15.22:36160) 2023-07-06T01:19:31.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.561+0000 7f0cec346700 1 Processor -- start 2023-07-06T01:19:31.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.561+0000 7f0cec346700 1 -- start start 2023-07-06T01:19:31.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.563+0000 7f0cec346700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f0ce4169940 0x7f0ce406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.563+0000 7f0cec346700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0ce408a220 0x7f0ce416d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.563+0000 7f0cec346700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce40cb430 con 0x7f0ce406d3a0 2023-07-06T01:19:31.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.563+0000 7f0cec346700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce40cab00 con 0x7f0ce408a220 2023-07-06T01:19:31.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.563+0000 7f0cec346700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce40cade0 con 0x7f0ce4169940 2023-07-06T01:19:31.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.563+0000 7f0cea8e3700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0ce408a220 0x7f0ce416d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.563+0000 7f0cea8e3700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0ce408a220 0x7f0ce416d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45408/0 (socket says 172.21.15.22:45408) 2023-07-06T01:19:31.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.563+0000 7f0cea8e3700 1 -- 172.21.15.22:0/3732172421 learned_addr learned my addr 172.21.15.22:0/3732172421 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.563+0000 7f0ce98e1700 1 --2- 172.21.15.22:0/3732172421 >> v2:172.21.15.103:3300/0 conn(0x7f0ce4169940 0x7f0ce406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7f35ece33700 1 -- 172.21.15.22:0/2429568530 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e4011830 con 0x7f35e8161010 2023-07-06T01:19:31.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7f35ece33700 1 -- 172.21.15.22:0/2429568530 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e4011990 con 0x7f35e8161010 2023-07-06T01:19:31.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.564+0000 7f35ece33700 1 -- 172.21.15.22:0/2429568530 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e4011d20 con 0x7f35e8161010 2023-07-06T01:19:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.565+0000 7f0cea8e3700 1 -- 172.21.15.22:0/3732172421 >> v2:172.21.15.103:3300/0 conn(0x7f0ce4169940 msgr2=0x7f0ce406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.565+0000 7f0cea8e3700 1 --2- 172.21.15.22:0/3732172421 >> v2:172.21.15.103:3300/0 conn(0x7f0ce4169940 0x7f0ce406b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.565+0000 7f0cea8e3700 1 -- 172.21.15.22:0/3732172421 >> v1:172.21.15.103:6789/0 conn(0x7f0ce406d3a0 legacy=0x7f0ce406d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.565+0000 7f0cea8e3700 1 -- 172.21.15.22:0/3732172421 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce40cb0c0 con 0x7f0ce408a220 2023-07-06T01:19:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.565+0000 7f0cea8e3700 1 --2- 172.21.15.22:0/3732172421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ce408a220 0x7f0ce416d4d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40cdfc0 tx=0x7f0cdc008190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28abce17ac5b832 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.565+0000 7f35f0099700 1 -- 172.21.15.22:0/2429568530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35e8161010 msgr2=0x7f35e806f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.565+0000 7f35f0099700 1 --2- 172.21.15.22:0/2429568530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35e8161010 0x7f35e806f910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009840 tx=0x7f35e4009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.565+0000 7f35f0099700 1 -- 172.21.15.22:0/2429568530 shutdown_connections 2023-07-06T01:19:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.565+0000 7f35f0099700 1 --2- 172.21.15.22:0/2429568530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35e8161010 0x7f35e806f910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.565+0000 7f35f0099700 1 --2- 172.21.15.22:0/2429568530 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35e80c2290 0x7f35e80ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.565+0000 7f35f0099700 1 -- 172.21.15.22:0/2429568530 >> 172.21.15.22:0/2429568530 conn(0x7f35e80b0710 msgr2=0x7f35e80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.565+0000 7f35f0099700 1 -- 172.21.15.22:0/2429568530 shutdown_connections 2023-07-06T01:19:31.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.565+0000 7fd91224a700 1 Processor -- start 2023-07-06T01:19:31.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.566+0000 7f0b9f520700 1 Processor -- start 2023-07-06T01:19:31.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.566+0000 7fade5a43700 1 -- 172.21.15.22:0/3403493750 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fade006d3a0 msgr2=0x7fade00adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.566+0000 7fade5a43700 1 --2- 172.21.15.22:0/3403493750 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fade006d3a0 0x7fade00adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.566+0000 7fade5a43700 1 -- 172.21.15.22:0/3403493750 >> v1:172.21.15.22:6789/0 conn(0x7fade00acdb0 legacy=0x7fade006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.566+0000 7fade5a43700 1 -- 172.21.15.22:0/3403493750 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade00cf730 con 0x7fade006d770 2023-07-06T01:19:31.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.566+0000 7f0b9f520700 1 -- start start 2023-07-06T01:19:31.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.566+0000 7fade5a43700 1 --2- 172.21.15.22:0/3403493750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade006d770 0x7fade00ac870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fadd00098f0 tx=0x7fadd0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1458d8ca589ea08f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.566+0000 7f35f0099700 1 -- 172.21.15.22:0/2429568530 wait complete. 2023-07-06T01:19:31.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.566+0000 7fade5242700 1 -- 172.21.15.22:0/3403493750 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000b8a0 con 0x7fade006d770 2023-07-06T01:19:31.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.566+0000 7fade5242700 1 -- 172.21.15.22:0/3403493750 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd000ba00 con 0x7fade006d770 2023-07-06T01:19:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.566+0000 7fade5242700 1 -- 172.21.15.22:0/3403493750 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000bd90 con 0x7fade006d770 2023-07-06T01:19:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fade84a8700 1 -- 172.21.15.22:0/3403493750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade006d770 msgr2=0x7fade00ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fade84a8700 1 --2- 172.21.15.22:0/3403493750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade006d770 0x7fade00ac870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fadd00098f0 tx=0x7fadd0009c00 comp rx=0 tx=0).stop 2023-07-06T01:19:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.566+0000 7fd91224a700 1 -- start start 2023-07-06T01:19:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fd91224a700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fd90c0c91d0 0x7fd90c0cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fd91224a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd90c06b9e0 0x7fd90c06be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fd91224a700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90c0cb1c0 con 0x7fd90c06d3a0 2023-07-06T01:19:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fd91224a700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90c0cb410 con 0x7fd90c0c91d0 2023-07-06T01:19:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fd91224a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c15d4e0 con 0x7fd90c06b9e0 2023-07-06T01:19:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7f0b9f520700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0b980adb20 0x7f0b9806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7f0b9f520700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b9806bf20 0x7f0b9806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7f0b9f520700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b9806c8e0 con 0x7f0b9806d3a0 2023-07-06T01:19:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7f0b9f520700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b981613a0 con 0x7f0b980adb20 2023-07-06T01:19:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7f0b9f520700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980cf450 con 0x7f0b9806bf20 2023-07-06T01:19:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fade84a8700 1 -- 172.21.15.22:0/3403493750 shutdown_connections 2023-07-06T01:19:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fade84a8700 1 --2- 172.21.15.22:0/3403493750 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fade006d3a0 0x7fade00adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fade84a8700 1 --2- 172.21.15.22:0/3403493750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade006d770 0x7fade00ac870 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fade84a8700 1 -- 172.21.15.22:0/3403493750 >> 172.21.15.22:0/3403493750 conn(0x7fade00b13a0 msgr2=0x7fade00ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fd90bfff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd90c06b9e0 0x7fd90c06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fd90bfff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd90c06b9e0 0x7fd90c06be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56558/0 (socket says 172.21.15.22:56558) 2023-07-06T01:19:31.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7fd90bfff700 1 -- 172.21.15.22:0/3511514692 learned_addr learned my addr 172.21.15.22:0/3511514692 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7f0ce90e0700 1 -- 172.21.15.22:0/3732172421 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc008d10 con 0x7f0ce408a220 2023-07-06T01:19:31.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7f0ce90e0700 1 -- 172.21.15.22:0/3732172421 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc008e70 con 0x7f0ce408a220 2023-07-06T01:19:31.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.567+0000 7f0ce90e0700 1 -- 172.21.15.22:0/3732172421 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc003c20 con 0x7f0ce408a220 2023-07-06T01:19:31.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:19:31.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7fd90bfff700 1 -- 172.21.15.22:0/3511514692 >> v2:172.21.15.103:3300/0 conn(0x7fd90c0c91d0 msgr2=0x7fd90c0cac80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T01:19:31.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7fd90bfff700 1 --2- 172.21.15.22:0/3511514692 >> v2:172.21.15.103:3300/0 conn(0x7fd90c0c91d0 0x7fd90c0cac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7fd90bfff700 1 -- 172.21.15.22:0/3511514692 >> v1:172.21.15.22:6789/0 conn(0x7fd90c06d3a0 legacy=0x7fd90c06d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7fd90bfff700 1 -- 172.21.15.22:0/3511514692 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c0cdd50 con 0x7fd90c06b9e0 2023-07-06T01:19:31.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7f35f0099700 1 Processor -- start 2023-07-06T01:19:31.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7f0b9cabb700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0b980adb20 0x7f0b9806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7f0b9cabb700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0b980adb20 0x7f0b9806b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45422/0 (socket says 172.21.15.22:45422) 2023-07-06T01:19:31.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7f0b9cabb700 1 -- 172.21.15.22:0/1539588437 learned_addr learned my addr 172.21.15.22:0/1539588437 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7f35f0099700 1 -- start start 2023-07-06T01:19:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7f0b9cabb700 1 -- 172.21.15.22:0/1539588437 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b9806bf20 msgr2=0x7f0b9806c3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T01:19:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7f0b9cabb700 1 --2- 172.21.15.22:0/1539588437 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b9806bf20 0x7f0b9806c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7f0b9cabb700 1 -- 172.21.15.22:0/1539588437 >> v1:172.21.15.22:6789/0 conn(0x7f0b9806d3a0 legacy=0x7f0b9806d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7f0b9cabb700 1 -- 172.21.15.22:0/1539588437 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b980cf730 con 0x7f0b980adb20 2023-07-06T01:19:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7fd90bfff700 1 --2- 172.21.15.22:0/3511514692 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd90c06b9e0 0x7fd90c06be60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd90400d370 tx=0x7fd90400d680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c9f5e5db35c74b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.568+0000 7fade84a8700 1 -- 172.21.15.22:0/3403493750 shutdown_connections 2023-07-06T01:19:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7fade84a8700 1 -- 172.21.15.22:0/3403493750 wait complete. 2023-07-06T01:19:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7f0b9cabb700 1 --2- 172.21.15.22:0/1539588437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b980adb20 0x7f0b9806b9e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009e50 tx=0x7f0b9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9695d766380b9a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7fd90a7fc700 1 -- 172.21.15.22:0/3511514692 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904010070 con 0x7fd90c06b9e0 2023-07-06T01:19:31.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7f7fa314c700 1 Processor -- start 2023-07-06T01:19:31.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7fd90a7fc700 1 -- 172.21.15.22:0/3511514692 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90400dcd0 con 0x7fd90c06b9e0 2023-07-06T01:19:31.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7fd90a7fc700 1 -- 172.21.15.22:0/3511514692 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904007d70 con 0x7fd90c06b9e0 2023-07-06T01:19:31.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7f0b8ffff700 1 -- 172.21.15.22:0/1539588437 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9000e070 con 0x7f0b980adb20 2023-07-06T01:19:31.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7f0b8ffff700 1 -- 172.21.15.22:0/1539588437 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9000b9c0 con 0x7f0b980adb20 2023-07-06T01:19:31.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7f0b8ffff700 1 -- 172.21.15.22:0/1539588437 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9000bd70 con 0x7f0b980adb20 2023-07-06T01:19:31.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7f35f0099700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35e80c2290 0x7f35e80c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7f35f0099700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35e80cd250 0x7f35e81642e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7f35f0099700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35e8161010 0x7f35e815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7f35f0099700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e8164820 con 0x7f35e8161010 2023-07-06T01:19:31.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7f35f0099700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80c42d0 con 0x7f35e80c2290 2023-07-06T01:19:31.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.569+0000 7f35f0099700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80c45b0 con 0x7f35e80cd250 2023-07-06T01:19:31.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7f35ed634700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35e80cd250 0x7f35e81642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7f35ee636700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35e8161010 0x7f35e815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7f35ee636700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35e8161010 0x7f35e815eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45432/0 (socket says 172.21.15.22:45432) 2023-07-06T01:19:31.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7f35ee636700 1 -- 172.21.15.22:0/2849901379 learned_addr learned my addr 172.21.15.22:0/2849901379 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7fd91224a700 1 -- 172.21.15.22:0/3511514692 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd90c06b9e0 msgr2=0x7fd90c06be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7fd91224a700 1 --2- 172.21.15.22:0/3511514692 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd90c06b9e0 0x7fd90c06be60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd90400d370 tx=0x7fd90400d680 comp rx=0 tx=0).stop 2023-07-06T01:19:31.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7f35ee636700 1 -- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35e80cd250 msgr2=0x7f35e81642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7f35ee636700 1 --2- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35e80cd250 0x7f35e81642e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7f35ee636700 1 -- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35e80c2290 msgr2=0x7f35e80c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7f35ee636700 1 --2- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35e80c2290 0x7f35e80c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7f35ee636700 1 -- 172.21.15.22:0/2849901379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e40094a0 con 0x7f35e8161010 2023-07-06T01:19:31.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7f35ee636700 1 --2- 172.21.15.22:0/2849901379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35e8161010 0x7f35e815eb40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f35e0002230 tx=0x7f35e000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7fade84a8700 1 Processor -- start 2023-07-06T01:19:31.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7fd91224a700 1 -- 172.21.15.22:0/3511514692 shutdown_connections 2023-07-06T01:19:31.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7fd91224a700 1 --2- 172.21.15.22:0/3511514692 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd90c06b9e0 0x7fd90c06be60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7fd91224a700 1 --2- 172.21.15.22:0/3511514692 >> v2:172.21.15.103:3300/0 conn(0x7fd90c0c91d0 0x7fd90c0cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7fd91224a700 1 -- 172.21.15.22:0/3511514692 >> 172.21.15.22:0/3511514692 conn(0x7fd90c0b13a0 msgr2=0x7fd90c0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7f0b9f520700 1 -- 172.21.15.22:0/1539588437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b980adb20 msgr2=0x7f0b9806b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7f0b9f520700 1 --2- 172.21.15.22:0/1539588437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b980adb20 0x7f0b9806b9e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009e50 tx=0x7f0b9000b330 comp rx=0 tx=0).stop 2023-07-06T01:19:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f0b9f520700 1 -- 172.21.15.22:0/1539588437 shutdown_connections 2023-07-06T01:19:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f0b9f520700 1 --2- 172.21.15.22:0/1539588437 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b9806bf20 0x7f0b9806c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f0b9f520700 1 --2- 172.21.15.22:0/1539588437 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b980adb20 0x7f0b9806b9e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f0b9f520700 1 -- 172.21.15.22:0/1539588437 >> 172.21.15.22:0/1539588437 conn(0x7f0b980b13a0 msgr2=0x7f0b980ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7fade84a8700 1 -- start start 2023-07-06T01:19:31.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f0b9f520700 1 -- 172.21.15.22:0/1539588437 shutdown_connections 2023-07-06T01:19:31.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f0b9f520700 1 -- 172.21.15.22:0/1539588437 wait complete. 2023-07-06T01:19:31.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f0e324cc700 1 Processor -- start 2023-07-06T01:19:31.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.570+0000 7f7fa314c700 1 -- start start 2023-07-06T01:19:31.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f7fa314c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7f9c160b50 0x7f7f9c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f7fa314c700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7f9c06a710 0x7f7f9c160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f7fa314c700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f9c070d30 con 0x7f7f9c0cbd00 2023-07-06T01:19:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f7fa314c700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9c0cf600 con 0x7f7f9c160b50 2023-07-06T01:19:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f7fa314c700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9c0cecd0 con 0x7f7f9c06a710 2023-07-06T01:19:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f7fa0ee8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7f9c160b50 0x7f7f9c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f7fa0ee8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7f9c160b50 0x7f7f9c0cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45436/0 (socket says 172.21.15.22:45436) 2023-07-06T01:19:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f7fa0ee8700 1 -- 172.21.15.22:0/2080457645 learned_addr learned my addr 172.21.15.22:0/2080457645 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f0cec346700 1 -- 172.21.15.22:0/3732172421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ce408a220 msgr2=0x7f0ce416d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f0cec346700 1 --2- 172.21.15.22:0/3732172421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ce408a220 0x7f0ce416d4d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40cdfc0 tx=0x7f0cdc008190 comp rx=0 tx=0).stop 2023-07-06T01:19:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7fade84a8700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fade006d3a0 0x7fade015cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7fade84a8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade006d770 0x7fade00c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7fade84a8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade00acdb0 0x7fade00c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7fade84a8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade00c2fb0 con 0x7fade006d770 2023-07-06T01:19:31.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7fade84a8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade00c3850 con 0x7fade00acdb0 2023-07-06T01:19:31.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7fade84a8700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade00c2340 con 0x7fade006d3a0 2023-07-06T01:19:31.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f0cec346700 1 -- 172.21.15.22:0/3732172421 shutdown_connections 2023-07-06T01:19:31.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f0cec346700 1 --2- 172.21.15.22:0/3732172421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ce408a220 0x7f0ce416d4d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f0cec346700 1 --2- 172.21.15.22:0/3732172421 >> v2:172.21.15.103:3300/0 conn(0x7f0ce4169940 0x7f0ce406b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f0cec346700 1 -- 172.21.15.22:0/3732172421 >> 172.21.15.22:0/3732172421 conn(0x7f0ce40b13a0 msgr2=0x7f0ce40ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f0cec346700 1 -- 172.21.15.22:0/3732172421 shutdown_connections 2023-07-06T01:19:31.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7fade5a43700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade006d770 0x7fade00c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7fade5a43700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade006d770 0x7fade00c5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45442/0 (socket says 172.21.15.22:45442) 2023-07-06T01:19:31.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7fade5a43700 1 -- 172.21.15.22:0/479859600 learned_addr learned my addr 172.21.15.22:0/479859600 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7fade6a45700 1 --2- 172.21.15.22:0/479859600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade00acdb0 0x7fade00c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.571+0000 7f0e324cc700 1 -- start start 2023-07-06T01:19:31.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f0e324cc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e2c15c940 0x7f0e2c0d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f0e324cc700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f0e2c0cd510 0x7f0e2c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f0e324cc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0e2c0d0fd0 0x7f0e2c1603d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f0e324cc700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2c09c8a0 con 0x7f0e2c0d0fd0 2023-07-06T01:19:31.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f0e324cc700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2c0d1450 con 0x7f0e2c0cd510 2023-07-06T01:19:31.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f0e324cc700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c15cd10 con 0x7f0e2c15c940 2023-07-06T01:19:31.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7fd91224a700 1 -- 172.21.15.22:0/3511514692 shutdown_connections 2023-07-06T01:19:31.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7f0e2bfff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e2c15c940 0x7f0e2c0d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f7fa0ee8700 1 -- 172.21.15.22:0/2080457645 >> v2:172.21.15.103:3300/0 conn(0x7f7f9c06a710 msgr2=0x7f7f9c160f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T01:19:31.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f7fa0ee8700 1 --2- 172.21.15.22:0/2080457645 >> v2:172.21.15.103:3300/0 conn(0x7f7f9c06a710 0x7f7f9c160f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f7fa0ee8700 1 -- 172.21.15.22:0/2080457645 >> v1:172.21.15.103:6789/0 conn(0x7f7f9c0cbd00 legacy=0x7f7f9c06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f7fa0ee8700 1 -- 172.21.15.22:0/2080457645 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c0cefb0 con 0x7f7f9c160b50 2023-07-06T01:19:31.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f7fa0ee8700 1 --2- 172.21.15.22:0/2080457645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c160b50 0x7f7f9c0cce50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0cd860 tx=0x7f7f9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5d664b12599bb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7f7f9b7fe700 1 -- 172.21.15.22:0/2080457645 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9400e070 con 0x7f7f9c160b50 2023-07-06T01:19:31.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7f7f9b7fe700 1 -- 172.21.15.22:0/2080457645 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f94003a20 con 0x7f7f9c160b50 2023-07-06T01:19:31.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7fade5a43700 1 -- 172.21.15.22:0/479859600 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fade006d3a0 msgr2=0x7fade015cd90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T01:19:31.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7fade5a43700 1 --2- 172.21.15.22:0/479859600 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fade006d3a0 0x7fade015cd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7fade5a43700 1 -- 172.21.15.22:0/479859600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade00acdb0 msgr2=0x7fade00c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7f0e2bfff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e2c15c940 0x7f0e2c0d2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56584/0 (socket says 172.21.15.22:56584) 2023-07-06T01:19:31.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7f0e2bfff700 1 -- 172.21.15.22:0/3968208134 learned_addr learned my addr 172.21.15.22:0/3968208134 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7fade5a43700 1 --2- 172.21.15.22:0/479859600 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fade00acdb0 0x7fade00c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7fade5a43700 1 -- 172.21.15.22:0/479859600 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd0009550 con 0x7fade006d770 2023-07-06T01:19:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7f7f9b7fe700 1 -- 172.21.15.22:0/2080457645 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94003dd0 con 0x7f7f9c160b50 2023-07-06T01:19:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7f0b9f520700 1 Processor -- start 2023-07-06T01:19:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f7fa314c700 1 -- 172.21.15.22:0/2080457645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c160b50 msgr2=0x7f7f9c0cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f7fa314c700 1 --2- 172.21.15.22:0/2080457645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c160b50 0x7f7f9c0cce50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0cd860 tx=0x7f7f9400b330 comp rx=0 tx=0).stop 2023-07-06T01:19:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f7fa314c700 1 -- 172.21.15.22:0/2080457645 shutdown_connections 2023-07-06T01:19:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f7fa314c700 1 --2- 172.21.15.22:0/2080457645 >> v2:172.21.15.103:3300/0 conn(0x7f7f9c06a710 0x7f7f9c160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f7fa314c700 1 --2- 172.21.15.22:0/2080457645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c160b50 0x7f7f9c0cce50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f7fa314c700 1 -- 172.21.15.22:0/2080457645 >> 172.21.15.22:0/2080457645 conn(0x7f7f9c06b110 msgr2=0x7f7f9c06b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7f35de7fc700 1 -- 172.21.15.22:0/2849901379 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e000ab70 con 0x7f35e8161010 2023-07-06T01:19:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7f35de7fc700 1 -- 172.21.15.22:0/2849901379 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e000acf0 con 0x7f35e8161010 2023-07-06T01:19:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7f35de7fc700 1 -- 172.21.15.22:0/2849901379 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0007310 con 0x7f35e8161010 2023-07-06T01:19:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7f35f0099700 1 -- 172.21.15.22:0/2849901379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e80c4890 con 0x7f35e8161010 2023-07-06T01:19:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7f35f0099700 1 -- 172.21.15.22:0/2849901379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e80c4e50 con 0x7f35e8161010 2023-07-06T01:19:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f35f0099700 1 -- 172.21.15.22:0/2849901379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35e81613e0 con 0x7f35e8161010 2023-07-06T01:19:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0e2b7fe700 1 --2- 172.21.15.22:0/3968208134 >> v2:172.21.15.103:3300/0 conn(0x7f0e2c0cd510 0x7f0e2c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0e30a69700 1 --2- 172.21.15.22:0/3968208134 >> v2:172.21.15.22:3300/0 conn(0x7f0e2c0d0fd0 0x7f0e2c1603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.572+0000 7f0cec346700 1 -- 172.21.15.22:0/3732172421 wait complete. 2023-07-06T01:19:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0cec346700 1 Processor -- start 2023-07-06T01:19:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0e2bfff700 1 -- 172.21.15.22:0/3968208134 >> v2:172.21.15.103:3300/0 conn(0x7f0e2c0cd510 msgr2=0x7f0e2c0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0e2bfff700 1 --2- 172.21.15.22:0/3968208134 >> v2:172.21.15.103:3300/0 conn(0x7f0e2c0cd510 0x7f0e2c0d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0e2bfff700 1 -- 172.21.15.22:0/3968208134 >> v2:172.21.15.22:3300/0 conn(0x7f0e2c0d0fd0 msgr2=0x7f0e2c1603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0e2bfff700 1 --2- 172.21.15.22:0/3968208134 >> v2:172.21.15.22:3300/0 conn(0x7f0e2c0d0fd0 0x7f0e2c1603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0e2bfff700 1 -- 172.21.15.22:0/3968208134 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0ce710 con 0x7f0e2c15c940 2023-07-06T01:19:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7fade5a43700 1 --2- 172.21.15.22:0/479859600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade006d770 0x7fade00c5c50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fadd0014040 tx=0x7fadd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7fd91224a700 1 -- 172.21.15.22:0/3511514692 wait complete. 2023-07-06T01:19:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f304a91e700 1 Processor -- start 2023-07-06T01:19:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0e30a69700 1 --2- 172.21.15.22:0/3968208134 >> v2:172.21.15.22:3300/0 conn(0x7f0e2c0d0fd0 0x7f0e2c1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.573+0000 7f0b9f520700 1 -- start start 2023-07-06T01:19:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0b9f520700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b9806bf20 0x7f0b981589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0b9f520700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b9806d3a0 0x7f0b980af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0b9f520700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b980adb20 0x7f0b981fe8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0b9f520700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980b2620 con 0x7f0b980adb20 2023-07-06T01:19:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0b9f520700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980b1cc0 con 0x7f0b9806d3a0 2023-07-06T01:19:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0b9f520700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980b1f60 con 0x7f0b9806bf20 2023-07-06T01:19:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f304a91e700 1 -- start start 2023-07-06T01:19:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0e2bfff700 1 --2- 172.21.15.22:0/3968208134 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e2c15c940 0x7f0e2c0d2a10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c068060 tx=0x7f0e2400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e26c0724be2c8b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f7fa314c700 1 -- 172.21.15.22:0/2080457645 shutdown_connections 2023-07-06T01:19:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.574+0000 7f0cec346700 1 -- start start 2023-07-06T01:19:31.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0e2affd700 1 -- 172.21.15.22:0/3968208134 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2400e070 con 0x7f0e2c15c940 2023-07-06T01:19:31.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0e2affd700 1 -- 172.21.15.22:0/3968208134 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2400b910 con 0x7f0e2c15c940 2023-07-06T01:19:31.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0e2affd700 1 -- 172.21.15.22:0/3968208134 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2400bcc0 con 0x7f0e2c15c940 2023-07-06T01:19:31.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7fd91224a700 1 Processor -- start 2023-07-06T01:19:31.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7fd91224a700 1 -- start start 2023-07-06T01:19:31.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0cec346700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0ce406d3a0 0x7f0ce4207870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0cec346700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ce408a220 0x7f0ce416af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0cec346700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ce4169940 0x7f0ce40c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0cec346700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce420baf0 con 0x7f0ce4169940 2023-07-06T01:19:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0cec346700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce416b880 con 0x7f0ce408a220 2023-07-06T01:19:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0cec346700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce420b4a0 con 0x7f0ce406d3a0 2023-07-06T01:19:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0cea0e2700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0ce406d3a0 0x7f0ce4207870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0cea0e2700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0ce406d3a0 0x7f0ce4207870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56596/0 (socket says 172.21.15.22:56596) 2023-07-06T01:19:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0cea0e2700 1 -- 172.21.15.22:0/3414194763 learned_addr learned my addr 172.21.15.22:0/3414194763 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0ce98e1700 1 --2- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ce408a220 0x7f0ce416af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0b9dabd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b980adb20 0x7f0b981fe8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0b9dabd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b980adb20 0x7f0b981fe8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45452/0 (socket says 172.21.15.22:45452) 2023-07-06T01:19:31.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.575+0000 7f0b9dabd700 1 -- 172.21.15.22:0/400902376 learned_addr learned my addr 172.21.15.22:0/400902376 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0e324cc700 1 -- 172.21.15.22:0/3968208134 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e2c15c940 msgr2=0x7f0e2c0d2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0e324cc700 1 --2- 172.21.15.22:0/3968208134 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e2c15c940 0x7f0e2c0d2a10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c068060 tx=0x7f0e2400b330 comp rx=0 tx=0).stop 2023-07-06T01:19:31.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0b9d2bc700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b9806bf20 0x7f0b981589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0b9cabb700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b9806d3a0 0x7f0b980af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7fadd6ffd700 1 -- 172.21.15.22:0/479859600 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd00038a0 con 0x7fade006d770 2023-07-06T01:19:31.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7fadd6ffd700 1 -- 172.21.15.22:0/479859600 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd001e070 con 0x7fade006d770 2023-07-06T01:19:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7fadd6ffd700 1 -- 172.21.15.22:0/479859600 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0003c40 con 0x7fade006d770 2023-07-06T01:19:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0cea0e2700 1 -- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ce408a220 msgr2=0x7f0ce416af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0cea0e2700 1 --2- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ce408a220 0x7f0ce416af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0cea0e2700 1 -- 172.21.15.22:0/3414194763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ce4169940 msgr2=0x7f0ce40c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0cea0e2700 1 --2- 172.21.15.22:0/3414194763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ce4169940 0x7f0ce40c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0cea0e2700 1 -- 172.21.15.22:0/3414194763 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc007b10 con 0x7f0ce406d3a0 2023-07-06T01:19:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0b9cabb700 1 -- 172.21.15.22:0/400902376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b9806bf20 msgr2=0x7f0b981589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0b9cabb700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0b9806bf20 0x7f0b981589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0b9cabb700 1 -- 172.21.15.22:0/400902376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b980adb20 msgr2=0x7f0b981fe8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0b9cabb700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b980adb20 0x7f0b981fe8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0b9cabb700 1 -- 172.21.15.22:0/400902376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b90009b30 con 0x7f0b9806d3a0 2023-07-06T01:19:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f304a91e700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f304406d3a0 0x7f30440adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f304a91e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f304406d770 0x7f30440ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f304a91e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30441613a0 con 0x7f30440acdb0 2023-07-06T01:19:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f304a91e700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30440ae060 con 0x7f304406d3a0 2023-07-06T01:19:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f304a91e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30440cf450 con 0x7f304406d770 2023-07-06T01:19:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f3043fff700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f304406d3a0 0x7f30440adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f3043fff700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f304406d3a0 0x7f30440adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:34188/0 (socket says 172.21.15.22:34188) 2023-07-06T01:19:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f3043fff700 1 -- 172.21.15.22:0/2335742650 learned_addr learned my addr 172.21.15.22:0/2335742650 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f30437fe700 1 --2- 172.21.15.22:0/2335742650 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f304406d770 0x7f30440ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f0cea0e2700 1 --2- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0ce406d3a0 0x7f0ce4207870 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400c4d0 tx=0x7f0cd400c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7f0b9cabb700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b9806d3a0 0x7f0b980af060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b90015000 tx=0x7f0b90004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7fade84a8700 1 -- 172.21.15.22:0/479859600 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade00c6860 con 0x7fade006d770 2023-07-06T01:19:31.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7fade84a8700 1 -- 172.21.15.22:0/479859600 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade00c34e0 con 0x7fade006d770 2023-07-06T01:19:31.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7fade84a8700 1 -- 172.21.15.22:0/479859600 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fade00cfa10 con 0x7fade006d770 2023-07-06T01:19:31.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7fd91224a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd90c06b9e0 0x7fd90c1583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd91224a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd90c06d3a0 0x7fd90c0cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd91224a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd90c0c91d0 0x7fd90c0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd91224a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0c8500 con 0x7fd90c0c91d0 2023-07-06T01:19:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd91224a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c1f2c40 con 0x7fd90c06b9e0 2023-07-06T01:19:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd91224a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c1f2f20 con 0x7fd90c06d3a0 2023-07-06T01:19:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd90affd700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd90c06d3a0 0x7fd90c0cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd90affd700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd90c06d3a0 0x7fd90c0cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56610/0 (socket says 172.21.15.22:56610) 2023-07-06T01:19:31.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd90affd700 1 -- 172.21.15.22:0/3667088037 learned_addr learned my addr 172.21.15.22:0/3667088037 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd90b7fe700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd90c06b9e0 0x7fd90c1583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd90bfff700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd90c0c91d0 0x7fd90c0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd90b7fe700 1 -- 172.21.15.22:0/3667088037 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd90c06d3a0 msgr2=0x7fd90c0cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd90b7fe700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd90c06d3a0 0x7fd90c0cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd90b7fe700 1 -- 172.21.15.22:0/3667088037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd90c0c91d0 msgr2=0x7fd90c0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd90b7fe700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd90c0c91d0 0x7fd90c0cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7fd90b7fe700 1 -- 172.21.15.22:0/3667088037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90400d050 con 0x7fd90c06b9e0 2023-07-06T01:19:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7f35de7fc700 1 -- 172.21.15.22:0/2849901379 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f35e0007470 con 0x7f35e8161010 2023-07-06T01:19:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7f35de7fc700 1 --2- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f35cc06adb0 0x7f35cc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7f35de7fc700 1 -- 172.21.15.22:0/2849901379 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f35e0089b40 con 0x7f35e8161010 2023-07-06T01:19:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f35ede35700 1 --2- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f35cc06adb0 0x7f35cc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f35de7fc700 1 --2- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] conn(0x7f35cc073660 0x7f35cc075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f35de7fc700 1 -- 172.21.15.22:0/2849901379 --> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35cc076120 con 0x7f35cc073660 2023-07-06T01:19:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f35de7fc700 1 -- 172.21.15.22:0/2849901379 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35e0089ef0 con 0x7f35e8161010 2023-07-06T01:19:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0e324cc700 1 -- 172.21.15.22:0/3968208134 shutdown_connections 2023-07-06T01:19:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0e324cc700 1 --2- 172.21.15.22:0/3968208134 >> v2:172.21.15.22:3300/0 conn(0x7f0e2c0d0fd0 0x7f0e2c1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0e324cc700 1 --2- 172.21.15.22:0/3968208134 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e2c15c940 0x7f0e2c0d2a10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0e324cc700 1 --2- 172.21.15.22:0/3968208134 >> v2:172.21.15.103:3300/0 conn(0x7f0e2c0cd510 0x7f0e2c0d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0e324cc700 1 -- 172.21.15.22:0/3968208134 >> 172.21.15.22:0/3968208134 conn(0x7f0e2c09c2e0 msgr2=0x7f0e2c0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7f0cdaffd700 1 -- 172.21.15.22:0/3414194763 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4011790 con 0x7f0ce406d3a0 2023-07-06T01:19:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7f0cdaffd700 1 -- 172.21.15.22:0/3414194763 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd40118f0 con 0x7f0ce406d3a0 2023-07-06T01:19:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7f0cdaffd700 1 -- 172.21.15.22:0/3414194763 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4011ca0 con 0x7f0ce406d3a0 2023-07-06T01:19:31.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0b8dffb700 1 -- 172.21.15.22:0/400902376 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90004450 con 0x7f0b9806d3a0 2023-07-06T01:19:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0b9f520700 1 -- 172.21.15.22:0/400902376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b980b2240 con 0x7f0b9806d3a0 2023-07-06T01:19:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0b8dffb700 1 -- 172.21.15.22:0/400902376 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b900045d0 con 0x7f0b9806d3a0 2023-07-06T01:19:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0b8dffb700 1 -- 172.21.15.22:0/400902376 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90004960 con 0x7f0b9806d3a0 2023-07-06T01:19:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0b9f520700 1 -- 172.21.15.22:0/400902376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b98158f30 con 0x7f0b9806d3a0 2023-07-06T01:19:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7fd90b7fe700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd90c06b9e0 0x7fd90c1583d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd90000c3e0 tx=0x7fd90000c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f35ed634700 1 --2- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] conn(0x7f35cc073660 0x7f35cc075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.579+0000 7f35ed634700 1 --2- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] conn(0x7f35cc073660 0x7f35cc075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0e324cc700 1 -- 172.21.15.22:0/3968208134 shutdown_connections 2023-07-06T01:19:31.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0e324cc700 1 -- 172.21.15.22:0/3968208134 wait complete. 2023-07-06T01:19:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0cec346700 1 -- 172.21.15.22:0/3414194763 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce420b780 con 0x7f0ce406d3a0 2023-07-06T01:19:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f0cec346700 1 -- 172.21.15.22:0/3414194763 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce416b650 con 0x7f0ce406d3a0 2023-07-06T01:19:31.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.579+0000 7f0cec346700 1 -- 172.21.15.22:0/3414194763 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ce40ca7b0 con 0x7f0ce406d3a0 2023-07-06T01:19:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.576+0000 7f7fa314c700 1 -- 172.21.15.22:0/2080457645 wait complete. 2023-07-06T01:19:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f7fa314c700 1 Processor -- start 2023-07-06T01:19:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f7fa314c700 1 -- start start 2023-07-06T01:19:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.579+0000 7f35ede35700 1 --2- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f35cc06adb0 0x7f35cc06d260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35e81640d0 tx=0x7f35e4010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.579+0000 7f7fa314c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f9c06a710 0x7f7f9c20fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f7fa314c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c0cbd00 0x7f7f9c1624c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f7fa314c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f9c160b50 0x7f7f9c0b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f7fa314c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c162ef0 con 0x7f7f9c0cbd00 2023-07-06T01:19:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f7fa314c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c1631d0 con 0x7f7f9c06a710 2023-07-06T01:19:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f7fa314c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c1634b0 con 0x7f7f9c160b50 2023-07-06T01:19:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7f30437fe700 1 -- 172.21.15.22:0/2335742650 >> v2:172.21.15.103:3300/0 conn(0x7f304406d3a0 msgr2=0x7f30440adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7f30437fe700 1 --2- 172.21.15.22:0/2335742650 >> v2:172.21.15.103:3300/0 conn(0x7f304406d3a0 0x7f30440adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7f30437fe700 1 -- 172.21.15.22:0/2335742650 >> v1:172.21.15.22:6789/0 conn(0x7f30440acdb0 legacy=0x7f304406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7f30437fe700 1 -- 172.21.15.22:0/2335742650 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30440cf730 con 0x7f304406d770 2023-07-06T01:19:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.577+0000 7f30437fe700 1 --2- 172.21.15.22:0/2335742650 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f304406d770 0x7f30440ac870 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3038009840 tx=0x7f3038009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48f566f4aaa0319c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f3042ffd700 1 -- 172.21.15.22:0/2335742650 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3038011830 con 0x7f304406d770 2023-07-06T01:19:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f3042ffd700 1 -- 172.21.15.22:0/2335742650 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3038011990 con 0x7f304406d770 2023-07-06T01:19:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.578+0000 7f3042ffd700 1 -- 172.21.15.22:0/2335742650 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3038011d00 con 0x7f304406d770 2023-07-06T01:19:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.579+0000 7f304a91e700 1 -- 172.21.15.22:0/2335742650 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f304406d770 msgr2=0x7f30440ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.579+0000 7f304a91e700 1 --2- 172.21.15.22:0/2335742650 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f304406d770 0x7f30440ac870 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3038009840 tx=0x7f3038009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.579+0000 7f304a91e700 1 -- 172.21.15.22:0/2335742650 shutdown_connections 2023-07-06T01:19:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.579+0000 7f304a91e700 1 --2- 172.21.15.22:0/2335742650 >> v2:172.21.15.103:3300/0 conn(0x7f304406d3a0 0x7f30440adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.579+0000 7f304a91e700 1 --2- 172.21.15.22:0/2335742650 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f304406d770 0x7f30440ac870 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.579+0000 7f304a91e700 1 -- 172.21.15.22:0/2335742650 >> 172.21.15.22:0/2335742650 conn(0x7f30440b13a0 msgr2=0x7f30440ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f304a91e700 1 -- 172.21.15.22:0/2335742650 shutdown_connections 2023-07-06T01:19:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.579+0000 7f0b837fe700 1 -- 172.21.15.22:0/400902376 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b70000fc0 con 0x7f0b9806d3a0 2023-07-06T01:19:31.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f304a91e700 1 -- 172.21.15.22:0/2335742650 wait complete. 2023-07-06T01:19:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7fd8f3fff700 1 -- 172.21.15.22:0/3667088037 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9000052d0 con 0x7fd90c06b9e0 2023-07-06T01:19:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7fd8f3fff700 1 -- 172.21.15.22:0/3667088037 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd900005450 con 0x7fd90c06b9e0 2023-07-06T01:19:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7fd8f3fff700 1 -- 172.21.15.22:0/3667088037 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9000038c0 con 0x7fd90c06b9e0 2023-07-06T01:19:31.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f7f9bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c0cbd00 0x7f7f9c1624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f7f9bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c0cbd00 0x7f7f9c1624c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45482/0 (socket says 172.21.15.22:45482) 2023-07-06T01:19:31.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f7f9bfff700 1 -- 172.21.15.22:0/1255999946 learned_addr learned my addr 172.21.15.22:0/1255999946 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f7fa16e9700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f9c160b50 0x7f7f9c0b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7fd91224a700 1 -- 172.21.15.22:0/3667088037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90c1f3200 con 0x7fd90c06b9e0 2023-07-06T01:19:31.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7fd91224a700 1 -- 172.21.15.22:0/3667088037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c158910 con 0x7fd90c06b9e0 2023-07-06T01:19:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f7fa0ee8700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f9c06a710 0x7f7f9c20fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7f7f9bfff700 1 -- 172.21.15.22:0/1255999946 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f9c160b50 msgr2=0x7f7f9c0b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7f7f9bfff700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f9c160b50 0x7f7f9c0b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7f7f9bfff700 1 -- 172.21.15.22:0/1255999946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f9c06a710 msgr2=0x7f7f9c20fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7f7f9bfff700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f9c06a710 0x7f7f9c20fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.641 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7f7f9bfff700 1 -- 172.21.15.22:0/1255999946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f94009b30 con 0x7f7f9c0cbd00 2023-07-06T01:19:31.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7fadd6ffd700 1 -- 172.21.15.22:0/479859600 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fadd0025480 con 0x7fade006d770 2023-07-06T01:19:31.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7fadd6ffd700 1 --2- 172.21.15.22:0/479859600 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fadc406ade0 0x7fadc406d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7fadd6ffd700 1 -- 172.21.15.22:0/479859600 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fadd0020070 con 0x7fade006d770 2023-07-06T01:19:31.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7fd91224a700 1 -- 172.21.15.22:0/3667088037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd90c1f2db0 con 0x7fd90c06b9e0 2023-07-06T01:19:31.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7f35de7fc700 1 -- 172.21.15.22:0/2849901379 <== osd.5 v2:172.21.15.103:6800/1893069563 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f35cc076120 con 0x7f35cc073660 2023-07-06T01:19:31.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7fadd6ffd700 1 --2- 172.21.15.22:0/479859600 >> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] conn(0x7fadc4073690 0x7fadc4075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7fadd6ffd700 1 -- 172.21.15.22:0/479859600 --> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fadc4076150 con 0x7fadc4073690 2023-07-06T01:19:31.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7fadd6ffd700 1 -- 172.21.15.22:0/479859600 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadd008b210 con 0x7fade006d770 2023-07-06T01:19:31.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7fade6a45700 1 --2- 172.21.15.22:0/479859600 >> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] conn(0x7fadc4073690 0x7fadc4075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f0e324cc700 1 Processor -- start 2023-07-06T01:19:31.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7f0e324cc700 1 -- start start 2023-07-06T01:19:31.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.580+0000 7f0cdaffd700 1 -- 172.21.15.22:0/3414194763 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0cd401c460 con 0x7f0ce406d3a0 2023-07-06T01:19:31.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7f0cdaffd700 1 --2- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0cc806ad80 0x7f0cc806d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7f0cdaffd700 1 -- 172.21.15.22:0/3414194763 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0cd4089bd0 con 0x7f0ce406d3a0 2023-07-06T01:19:31.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7f0cdaffd700 1 --2- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] conn(0x7f0cc8073630 0x7f0cc8075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7f0cdaffd700 1 -- 172.21.15.22:0/3414194763 --> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cc80760f0 con 0x7f0cc8073630 2023-07-06T01:19:31.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7f0cdaffd700 1 -- 172.21.15.22:0/3414194763 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd4089f80 con 0x7f0ce406d3a0 2023-07-06T01:19:31.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f0cea8e3700 1 --2- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] conn(0x7f0cc8073630 0x7f0cc8075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.581+0000 7f7f9bfff700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c0cbd00 0x7f7f9c1624c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c650 tx=0x7f7f8c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f304a91e700 1 Processor -- start 2023-07-06T01:19:31.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7fade6a45700 1 --2- 172.21.15.22:0/479859600 >> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] conn(0x7fadc4073690 0x7fadc4075aa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f7f997fa700 1 -- 172.21.15.22:0/1255999946 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c0054e0 con 0x7f7f9c0cbd00 2023-07-06T01:19:31.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f7f997fa700 1 -- 172.21.15.22:0/1255999946 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c003680 con 0x7f7f9c0cbd00 2023-07-06T01:19:31.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f7f997fa700 1 -- 172.21.15.22:0/1255999946 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c003a30 con 0x7f7f9c0cbd00 2023-07-06T01:19:31.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f0cea8e3700 1 --2- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] conn(0x7f0cc8073630 0x7f0cc8075a40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7fade6244700 1 --2- 172.21.15.22:0/479859600 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fadc406ade0 0x7fadc406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f0e324cc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e2c0cd510 0x7f0e2c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f0e324cc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e2c0d0fd0 0x7f0e2c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f0e324cc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2c15c940 0x7f0e2c1fe910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f0e324cc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0b49b0 con 0x7f0e2c15c940 2023-07-06T01:19:31.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f0e324cc700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c15f320 con 0x7f0e2c0d0fd0 2023-07-06T01:19:31.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f0e324cc700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c15f610 con 0x7f0e2c0cd510 2023-07-06T01:19:31.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f0e2bfff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e2c0cd510 0x7f0e2c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f0e2bfff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e2c0cd510 0x7f0e2c0b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56632/0 (socket says 172.21.15.22:56632) 2023-07-06T01:19:31.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f0e2bfff700 1 -- 172.21.15.22:0/3448000391 learned_addr learned my addr 172.21.15.22:0/3448000391 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.582+0000 7f0e2b7fe700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e2c0d0fd0 0x7f0e2c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f0e30a69700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2c15c940 0x7f0e2c1fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f0ce98e1700 1 --2- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0cc806ad80 0x7f0cc806d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7fadd6ffd700 1 -- 172.21.15.22:0/479859600 <== osd.2 v2:172.21.15.22:6816/579920713 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fadc4076150 con 0x7fadc4073690 2023-07-06T01:19:31.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f7fa314c700 1 -- 172.21.15.22:0/1255999946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9c163790 con 0x7f7f9c0cbd00 2023-07-06T01:19:31.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f7fa314c700 1 -- 172.21.15.22:0/1255999946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9c164440 con 0x7f7f9c0cbd00 2023-07-06T01:19:31.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f0cdaffd700 1 -- 172.21.15.22:0/3414194763 <== osd.7 v2:172.21.15.103:6816/1845979485 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0cc80760f0 con 0x7f0cc8073630 2023-07-06T01:19:31.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7fade6244700 1 --2- 172.21.15.22:0/479859600 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fadc406ade0 0x7fadc406d290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faddc009690 tx=0x7faddc006c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f0ce98e1700 1 --2- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0cc806ad80 0x7f0cc806d230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40c8ab0 tx=0x7f0ce0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f0b8dffb700 1 -- 172.21.15.22:0/400902376 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0b9001c440 con 0x7f0b9806d3a0 2023-07-06T01:19:31.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f0b8dffb700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0b7c06acb0 0x7f0b7c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f0b8dffb700 1 -- 172.21.15.22:0/400902376 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0b9008b290 con 0x7f0b9806d3a0 2023-07-06T01:19:31.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.584+0000 7f0b8dffb700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] conn(0x7f0b7c0733e0 0x7f0b7c0757f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.584+0000 7f0b8dffb700 1 -- 172.21.15.22:0/400902376 --> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b7c075ea0 con 0x7f0b7c0733e0 2023-07-06T01:19:31.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.584+0000 7f0b8dffb700 1 -- 172.21.15.22:0/400902376 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b9008b640 con 0x7f0b9806d3a0 2023-07-06T01:19:31.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f0e2b7fe700 1 -- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e2c0cd510 msgr2=0x7f0e2c0b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f0e2b7fe700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e2c0cd510 0x7f0e2c0b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f0e2b7fe700 1 -- 172.21.15.22:0/3448000391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2c15c940 msgr2=0x7f0e2c1fe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f0e2b7fe700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e2c15c940 0x7f0e2c1fe910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.583+0000 7f0e2b7fe700 1 -- 172.21.15.22:0/3448000391 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e24009b30 con 0x7f0e2c0d0fd0 2023-07-06T01:19:31.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.584+0000 7f0e2b7fe700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e2c0d0fd0 0x7f0e2c159c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00e870 tx=0x7f0e1c00eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.584+0000 7f0b9d2bc700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0b7c06acb0 0x7f0b7c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.584+0000 7f0b9dabd700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] conn(0x7f0b7c0733e0 0x7f0b7c0757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.584+0000 7f7fa314c700 1 -- 172.21.15.22:0/1255999946 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f9c0cd510 con 0x7f7f9c0cbd00 2023-07-06T01:19:31.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.585+0000 7f0b9dabd700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] conn(0x7f0b7c0733e0 0x7f0b7c0757f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.584+0000 7f304a91e700 1 -- start start 2023-07-06T01:19:31.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.585+0000 7f304a91e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f304406d3a0 0x7f304415b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.585+0000 7f304a91e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f304406d770 0x7f30441fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.585+0000 7f304a91e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30440acdb0 0x7f3044157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.585+0000 7f304a91e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30441593e0 con 0x7f30440acdb0 2023-07-06T01:19:31.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.585+0000 7f304a91e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30440c6b60 con 0x7f304406d3a0 2023-07-06T01:19:31.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.585+0000 7f304a91e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3044158590 con 0x7f304406d770 2023-07-06T01:19:31.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.585+0000 7f0e28ff9700 1 -- 172.21.15.22:0/3448000391 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c013680 con 0x7f0e2c0d0fd0 2023-07-06T01:19:31.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.585+0000 7f0e28ff9700 1 -- 172.21.15.22:0/3448000391 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c0137e0 con 0x7f0e2c0d0fd0 2023-07-06T01:19:31.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.585+0000 7f0e28ff9700 1 -- 172.21.15.22:0/3448000391 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c013b50 con 0x7f0e2c0d0fd0 2023-07-06T01:19:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.585+0000 7f3043fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f304406d3a0 0x7f304415b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.586+0000 7f3048ebb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30440acdb0 0x7f3044157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.586+0000 7f3048ebb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30440acdb0 0x7f3044157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45506/0 (socket says 172.21.15.22:45506) 2023-07-06T01:19:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.586+0000 7f3048ebb700 1 -- 172.21.15.22:0/229179646 learned_addr learned my addr 172.21.15.22:0/229179646 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.586+0000 7f0e324cc700 1 -- 172.21.15.22:0/3448000391 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c15f8f0 con 0x7f0e2c0d0fd0 2023-07-06T01:19:31.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.586+0000 7f0e324cc700 1 -- 172.21.15.22:0/3448000391 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c1fee50 con 0x7f0e2c0d0fd0 2023-07-06T01:19:31.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.587+0000 7f0e324cc700 1 -- 172.21.15.22:0/3448000391 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0e2c0cecd0 con 0x7f0e2c0d0fd0 2023-07-06T01:19:31.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.587+0000 7f7f997fa700 1 -- 172.21.15.22:0/1255999946 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f8c003b90 con 0x7f7f9c0cbd00 2023-07-06T01:19:31.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.586+0000 7f3048ebb700 1 -- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f304406d770 msgr2=0x7f30441fffe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T01:19:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.586+0000 7f3048ebb700 1 --2- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f304406d770 0x7f30441fffe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.586+0000 7f3048ebb700 1 -- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f304406d3a0 msgr2=0x7f304415b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.586+0000 7f3048ebb700 1 --2- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f304406d3a0 0x7f304415b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.586+0000 7f3048ebb700 1 -- 172.21.15.22:0/229179646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30380094a0 con 0x7f30440acdb0 2023-07-06T01:19:31.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.587+0000 7f3048ebb700 1 --2- 172.21.15.22:0/229179646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30440acdb0 0x7f3044157500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f303c00cbc0 tx=0x7f303c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.587+0000 7f0b9d2bc700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0b7c06acb0 0x7f0b7c06d160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0b88007fd0 tx=0x7f0b88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.587+0000 7f3040ff9700 1 -- 172.21.15.22:0/229179646 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c003c00 con 0x7f30440acdb0 2023-07-06T01:19:31.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.587+0000 7f7f997fa700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f8806ade0 0x7f7f8806d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.587+0000 7f3040ff9700 1 -- 172.21.15.22:0/229179646 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303c003d60 con 0x7f30440acdb0 2023-07-06T01:19:31.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.587+0000 7f3040ff9700 1 -- 172.21.15.22:0/229179646 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c00ab30 con 0x7f30440acdb0 2023-07-06T01:19:31.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.587+0000 7f7f997fa700 1 -- 172.21.15.22:0/1255999946 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7f8c089150 con 0x7f7f9c0cbd00 2023-07-06T01:19:31.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.587+0000 7f7fa0ee8700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f8806ade0 0x7f7f8806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.587+0000 7f7f997fa700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] conn(0x7f7f88073690 0x7f7f88075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.587+0000 7f7f997fa700 1 -- 172.21.15.22:0/1255999946 --> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f88076150 con 0x7f7f88073690 2023-07-06T01:19:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.587+0000 7f7f997fa700 1 -- 172.21.15.22:0/1255999946 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f8c089500 con 0x7f7f9c0cbd00 2023-07-06T01:19:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.588+0000 7f7fa0ee8700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f8806ade0 0x7f7f8806d290 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400b370 tx=0x7f7f94009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.588+0000 7f0e28ff9700 1 -- 172.21.15.22:0/3448000391 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e1c013cb0 con 0x7f0e2c0d0fd0 2023-07-06T01:19:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.588+0000 7f304a91e700 1 -- 172.21.15.22:0/229179646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3044200520 con 0x7f30440acdb0 2023-07-06T01:19:31.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.588+0000 7f304a91e700 1 -- 172.21.15.22:0/229179646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3044157fd0 con 0x7f30440acdb0 2023-07-06T01:19:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.589+0000 7f0e28ff9700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0e0c06ad30 0x7f0e0c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.589+0000 7f0e28ff9700 1 -- 172.21.15.22:0/3448000391 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e1c0894a0 con 0x7f0e2c0d0fd0 2023-07-06T01:19:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.589+0000 7f0e2bfff700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0e0c06ad30 0x7f0e0c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.589+0000 7f0e28ff9700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] conn(0x7f0e0c0735e0 0x7f0e0c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.589+0000 7f0e28ff9700 1 -- 172.21.15.22:0/3448000391 --> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0e0c0760a0 con 0x7f0e0c0735e0 2023-07-06T01:19:31.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.589+0000 7f0e28ff9700 1 -- 172.21.15.22:0/3448000391 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e1c089810 con 0x7f0e2c0d0fd0 2023-07-06T01:19:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.589+0000 7f0e30a69700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] conn(0x7f0e0c0735e0 0x7f0e0c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.589+0000 7f0e2bfff700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0e0c06ad30 0x7f0e0c06d1e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009440 tx=0x7f0e24009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.590+0000 7f0e30a69700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] conn(0x7f0e0c0735e0 0x7f0e0c0759f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.590+0000 7f0b8dffb700 1 -- 172.21.15.22:0/400902376 <== osd.0 v2:172.21.15.22:6808/683380483 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0b7c075ea0 con 0x7f0b7c0733e0 2023-07-06T01:19:31.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.590+0000 7fd8f3fff700 1 -- 172.21.15.22:0/3667088037 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd900003a20 con 0x7fd90c06b9e0 2023-07-06T01:19:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.590+0000 7f0e28ff9700 1 -- 172.21.15.22:0/3448000391 <== osd.6 v2:172.21.15.103:6808/2564178583 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0e0c0760a0 con 0x7f0e0c0735e0 2023-07-06T01:19:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.591+0000 7f3040ff9700 1 -- 172.21.15.22:0/229179646 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f303c00ac90 con 0x7f30440acdb0 2023-07-06T01:19:31.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.591+0000 7fd8f3fff700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd8f806adb0 0x7fd8f806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.591+0000 7fd8f3fff700 1 -- 172.21.15.22:0/3667088037 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd900088f30 con 0x7fd90c06b9e0 2023-07-06T01:19:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.591+0000 7fd8f3fff700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] conn(0x7fd8f8073660 0x7fd8f8075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.591+0000 7fd8f3fff700 1 -- 172.21.15.22:0/3667088037 --> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8f8076120 con 0x7fd8f8073660 2023-07-06T01:19:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.591+0000 7fd8f3fff700 1 -- 172.21.15.22:0/3667088037 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9000892e0 con 0x7fd90c06b9e0 2023-07-06T01:19:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.591+0000 7fd90bfff700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] conn(0x7fd8f8073660 0x7fd8f8075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.591+0000 7fd90affd700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd8f806adb0 0x7fd8f806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.592+0000 7f3040ff9700 1 --2- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f303006adb0 0x7f303006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.592+0000 7fd90bfff700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] conn(0x7fd8f8073660 0x7fd8f8075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.592+0000 7fd90affd700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd8f806adb0 0x7fd8f806d260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd90c1f2900 tx=0x7fd8fc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.592+0000 7f3040ff9700 1 -- 172.21.15.22:0/229179646 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f303c089510 con 0x7f30440acdb0 2023-07-06T01:19:31.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.592+0000 7f3043fff700 1 --2- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f303006adb0 0x7f303006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.592+0000 7f7fa16e9700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] conn(0x7f7f88073690 0x7f7f88075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.593+0000 7f304a91e700 1 --2- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] conn(0x7f3044158160 0x7f30440c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.593+0000 7f304a91e700 1 -- 172.21.15.22:0/229179646 --> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30440cc7f0 con 0x7f3044158160 2023-07-06T01:19:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.593+0000 7fd8f3fff700 1 -- 172.21.15.22:0/3667088037 <== osd.3 v2:172.21.15.22:6824/3305852500 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd8f8076120 con 0x7fd8f8073660 2023-07-06T01:19:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.593+0000 7f30437fe700 1 --2- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] conn(0x7f3044158160 0x7f30440c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.594+0000 7f30437fe700 1 --2- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] conn(0x7f3044158160 0x7f30440c9e60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.593+0000 7f7fa16e9700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] conn(0x7f7f88073690 0x7f7f88075aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.594+0000 7f7f997fa700 1 -- 172.21.15.22:0/1255999946 <== osd.1 v2:172.21.15.22:6800/2676174112 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7f88076150 con 0x7f7f88073690 2023-07-06T01:19:31.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.595+0000 7f3040ff9700 1 -- 172.21.15.22:0/229179646 <== osd.4 v2:172.21.15.103:6824/3712184942 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3038011630 con 0x7f3044158160 2023-07-06T01:19:31.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.603+0000 7f3043fff700 1 --2- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f303006adb0 0x7f303006d260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f303400d2d0 tx=0x7f3034008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.604+0000 7f35f0099700 1 -- 172.21.15.22:0/2849901379 --> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f35e80c4720 con 0x7f35cc073660 2023-07-06T01:19:31.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.605+0000 7f35de7fc700 1 -- 172.21.15.22:0/2849901379 <== osd.5 v2:172.21.15.103:6800/1893069563 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f35e80c4720 con 0x7f35cc073660 2023-07-06T01:19:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.612+0000 7f35f0099700 1 -- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] conn(0x7f35cc073660 msgr2=0x7f35cc075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.612+0000 7f35f0099700 1 --2- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] conn(0x7f35cc073660 0x7f35cc075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.612+0000 7f35f0099700 1 -- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f35cc06adb0 msgr2=0x7f35cc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.612+0000 7f35f0099700 1 --2- 172.21.15.22:0/2849901379 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f35cc06adb0 0x7f35cc06d260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35e81640d0 tx=0x7f35e4010000 comp rx=0 tx=0).stop 2023-07-06T01:19:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.612+0000 7f35f0099700 1 -- 172.21.15.22:0/2849901379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35e8161010 msgr2=0x7f35e815eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.612+0000 7f35f0099700 1 --2- 172.21.15.22:0/2849901379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f35e8161010 0x7f35e815eb40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f35e0002230 tx=0x7f35e000cf50 comp rx=0 tx=0).stop 2023-07-06T01:19:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.612+0000 7f35ee636700 1 -- 172.21.15.22:0/2849901379 reap_dead start 2023-07-06T01:19:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.612+0000 7f35f0099700 1 -- 172.21.15.22:0/2849901379 shutdown_connections 2023-07-06T01:19:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.612+0000 7f35f0099700 1 -- 172.21.15.22:0/2849901379 >> 172.21.15.22:0/2849901379 conn(0x7f35e80b0710 msgr2=0x7f35e8160040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.613+0000 7f35f0099700 1 -- 172.21.15.22:0/2849901379 shutdown_connections 2023-07-06T01:19:31.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.613+0000 7f35f0099700 1 -- 172.21.15.22:0/2849901379 wait complete. 2023-07-06T01:19:31.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.612+0000 7fade84a8700 1 -- 172.21.15.22:0/479859600 --> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fade00cc7f0 con 0x7fadc4073690 2023-07-06T01:19:31.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.613+0000 7fadd6ffd700 1 -- 172.21.15.22:0/479859600 <== osd.2 v2:172.21.15.22:6816/579920713 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fadd8001df0 con 0x7fadc4073690 2023-07-06T01:19:31.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.614+0000 7fade84a8700 1 -- 172.21.15.22:0/479859600 >> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] conn(0x7fadc4073690 msgr2=0x7fadc4075aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.614+0000 7fade84a8700 1 --2- 172.21.15.22:0/479859600 >> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] conn(0x7fadc4073690 0x7fadc4075aa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.614+0000 7fade84a8700 1 -- 172.21.15.22:0/479859600 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fadc406ade0 msgr2=0x7fadc406d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.614+0000 7fade84a8700 1 --2- 172.21.15.22:0/479859600 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fadc406ade0 0x7fadc406d290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faddc009690 tx=0x7faddc006c10 comp rx=0 tx=0).stop 2023-07-06T01:19:31.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.614+0000 7fade84a8700 1 -- 172.21.15.22:0/479859600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade006d770 msgr2=0x7fade00c5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.614+0000 7fade84a8700 1 --2- 172.21.15.22:0/479859600 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fade006d770 0x7fade00c5c50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fadd0014040 tx=0x7fadd0004750 comp rx=0 tx=0).stop 2023-07-06T01:19:31.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.614+0000 7fade6a45700 1 -- 172.21.15.22:0/479859600 reap_dead start 2023-07-06T01:19:31.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.615+0000 7fade84a8700 1 -- 172.21.15.22:0/479859600 shutdown_connections 2023-07-06T01:19:31.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.615+0000 7fade84a8700 1 -- 172.21.15.22:0/479859600 >> 172.21.15.22:0/479859600 conn(0x7fade00b13a0 msgr2=0x7fade01601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.615+0000 7fade84a8700 1 -- 172.21.15.22:0/479859600 shutdown_connections 2023-07-06T01:19:31.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.615+0000 7fade84a8700 1 -- 172.21.15.22:0/479859600 wait complete. 2023-07-06T01:19:31.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.616+0000 7f0cec346700 1 -- 172.21.15.22:0/3414194763 --> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ce420b300 con 0x7f0cc8073630 2023-07-06T01:19:31.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.617+0000 7f0cdaffd700 1 -- 172.21.15.22:0/3414194763 <== osd.7 v2:172.21.15.103:6816/1845979485 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0ce420b300 con 0x7f0cc8073630 2023-07-06T01:19:31.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.619+0000 7f0cec346700 1 -- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] conn(0x7f0cc8073630 msgr2=0x7f0cc8075a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.619+0000 7f0cec346700 1 --2- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] conn(0x7f0cc8073630 0x7f0cc8075a40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.619+0000 7f0cec346700 1 -- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0cc806ad80 msgr2=0x7f0cc806d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.619+0000 7f0cec346700 1 --2- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0cc806ad80 0x7f0cc806d230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40c8ab0 tx=0x7f0ce0008040 comp rx=0 tx=0).stop 2023-07-06T01:19:31.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.619+0000 7f0cec346700 1 -- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0ce406d3a0 msgr2=0x7f0ce4207870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.619+0000 7f0cec346700 1 --2- 172.21.15.22:0/3414194763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0ce406d3a0 0x7f0ce4207870 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400c4d0 tx=0x7f0cd400c7e0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.620+0000 7f0cea8e3700 1 -- 172.21.15.22:0/3414194763 reap_dead start 2023-07-06T01:19:31.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.622+0000 7f0cec346700 1 -- 172.21.15.22:0/3414194763 shutdown_connections 2023-07-06T01:19:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.622+0000 7f0cec346700 1 -- 172.21.15.22:0/3414194763 >> 172.21.15.22:0/3414194763 conn(0x7f0ce40b13a0 msgr2=0x7f0ce4168fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.622+0000 7f0b837fe700 1 -- 172.21.15.22:0/400902376 --> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0b70002d40 con 0x7f0b7c0733e0 2023-07-06T01:19:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.622+0000 7f0cec346700 1 -- 172.21.15.22:0/3414194763 shutdown_connections 2023-07-06T01:19:31.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.622+0000 7f0cec346700 1 -- 172.21.15.22:0/3414194763 wait complete. 2023-07-06T01:19:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.623+0000 7f0b8dffb700 1 -- 172.21.15.22:0/400902376 <== osd.0 v2:172.21.15.22:6808/683380483 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0b70002d40 con 0x7f0b7c0733e0 2023-07-06T01:19:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.626+0000 7fd91224a700 1 -- 172.21.15.22:0/3667088037 --> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd90c15ca00 con 0x7fd8f8073660 2023-07-06T01:19:31.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.626+0000 7f0b837fe700 1 -- 172.21.15.22:0/400902376 >> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] conn(0x7f0b7c0733e0 msgr2=0x7f0b7c0757f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.626+0000 7f0b837fe700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] conn(0x7f0b7c0733e0 0x7f0b7c0757f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.626+0000 7f0e324cc700 1 -- 172.21.15.22:0/3448000391 --> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0e2c0ce9f0 con 0x7f0e0c0735e0 2023-07-06T01:19:31.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.626+0000 7f0b837fe700 1 -- 172.21.15.22:0/400902376 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0b7c06acb0 msgr2=0x7f0b7c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.626+0000 7f0b837fe700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0b7c06acb0 0x7f0b7c06d160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0b88007fd0 tx=0x7f0b88009040 comp rx=0 tx=0).stop 2023-07-06T01:19:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.626+0000 7f0b837fe700 1 -- 172.21.15.22:0/400902376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b9806d3a0 msgr2=0x7f0b980af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.626+0000 7f0b837fe700 1 --2- 172.21.15.22:0/400902376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b9806d3a0 0x7f0b980af060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b90015000 tx=0x7f0b90004ad0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.626+0000 7f0b9dabd700 1 -- 172.21.15.22:0/400902376 reap_dead start 2023-07-06T01:19:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.627+0000 7f0b837fe700 1 -- 172.21.15.22:0/400902376 shutdown_connections 2023-07-06T01:19:31.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.627+0000 7f0b837fe700 1 -- 172.21.15.22:0/400902376 >> 172.21.15.22:0/400902376 conn(0x7f0b980b13a0 msgr2=0x7f0b9806d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.627+0000 7fd8f3fff700 1 -- 172.21.15.22:0/3667088037 <== osd.3 v2:172.21.15.22:6824/3305852500 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd9040071f0 con 0x7fd8f8073660 2023-07-06T01:19:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.627+0000 7f0b837fe700 1 -- 172.21.15.22:0/400902376 shutdown_connections 2023-07-06T01:19:31.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.627+0000 7f0b837fe700 1 -- 172.21.15.22:0/400902376 wait complete. 2023-07-06T01:19:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.627+0000 7f0e28ff9700 1 -- 172.21.15.22:0/3448000391 <== osd.6 v2:172.21.15.103:6808/2564178583 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0e14009a20 con 0x7f0e0c0735e0 2023-07-06T01:19:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.627+0000 7f7fa314c700 1 -- 172.21.15.22:0/1255999946 --> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f9c0cf290 con 0x7f7f88073690 2023-07-06T01:19:31.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.628+0000 7fd91224a700 1 -- 172.21.15.22:0/3667088037 >> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] conn(0x7fd8f8073660 msgr2=0x7fd8f8075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.628+0000 7fd91224a700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] conn(0x7fd8f8073660 0x7fd8f8075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.628+0000 7fd91224a700 1 -- 172.21.15.22:0/3667088037 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd8f806adb0 msgr2=0x7fd8f806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.628+0000 7fd91224a700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd8f806adb0 0x7fd8f806d260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd90c1f2900 tx=0x7fd8fc009190 comp rx=0 tx=0).stop 2023-07-06T01:19:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.628+0000 7fd91224a700 1 -- 172.21.15.22:0/3667088037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd90c06b9e0 msgr2=0x7fd90c1583d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.628+0000 7fd91224a700 1 --2- 172.21.15.22:0/3667088037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd90c06b9e0 0x7fd90c1583d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd90000c3e0 tx=0x7fd90000c6f0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.628+0000 7fd90bfff700 1 -- 172.21.15.22:0/3667088037 reap_dead start 2023-07-06T01:19:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.628+0000 7f7f997fa700 1 -- 172.21.15.22:0/1255999946 <== osd.1 v2:172.21.15.22:6800/2676174112 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7f9000a380 con 0x7f7f88073690 2023-07-06T01:19:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.628+0000 7fd91224a700 1 -- 172.21.15.22:0/3667088037 shutdown_connections 2023-07-06T01:19:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.628+0000 7fd91224a700 1 -- 172.21.15.22:0/3667088037 >> 172.21.15.22:0/3667088037 conn(0x7fd90c0b13a0 msgr2=0x7fd90c06d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.629+0000 7f7fa314c700 1 -- 172.21.15.22:0/1255999946 >> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] conn(0x7f7f88073690 msgr2=0x7f7f88075aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.629+0000 7f7fa314c700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] conn(0x7f7f88073690 0x7f7f88075aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.630+0000 7f0e324cc700 1 -- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] conn(0x7f0e0c0735e0 msgr2=0x7f0e0c0759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.630+0000 7f0e324cc700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] conn(0x7f0e0c0735e0 0x7f0e0c0759f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.630+0000 7f0e324cc700 1 -- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0e0c06ad30 msgr2=0x7f0e0c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.630+0000 7f0e324cc700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0e0c06ad30 0x7f0e0c06d1e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009440 tx=0x7f0e24009000 comp rx=0 tx=0).stop 2023-07-06T01:19:31.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.630+0000 7f0e324cc700 1 -- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e2c0d0fd0 msgr2=0x7f0e2c159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.630+0000 7f0e324cc700 1 --2- 172.21.15.22:0/3448000391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e2c0d0fd0 0x7f0e2c159c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00e870 tx=0x7f0e1c00eb80 comp rx=0 tx=0).stop 2023-07-06T01:19:31.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.630+0000 7f0e30a69700 1 -- 172.21.15.22:0/3448000391 reap_dead start 2023-07-06T01:19:31.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.631+0000 7f0e324cc700 1 -- 172.21.15.22:0/3448000391 shutdown_connections 2023-07-06T01:19:31.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.631+0000 7f0e324cc700 1 -- 172.21.15.22:0/3448000391 >> 172.21.15.22:0/3448000391 conn(0x7f0e2c09c2e0 msgr2=0x7f0e2c0afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.691 INFO:teuthology.orchestra.run.smithi022.stdout:55834574853 2023-07-06T01:19:31.692 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-06T01:19:31.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.630+0000 7fd91224a700 1 -- 172.21.15.22:0/3667088037 shutdown_connections 2023-07-06T01:19:31.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.633+0000 7fd91224a700 1 -- 172.21.15.22:0/3667088037 wait complete. 2023-07-06T01:19:31.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.634+0000 7f7fa314c700 1 -- 172.21.15.22:0/1255999946 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f8806ade0 msgr2=0x7f7f8806d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.634+0000 7f7fa314c700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f8806ade0 0x7f7f8806d290 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400b370 tx=0x7f7f94009000 comp rx=0 tx=0).stop 2023-07-06T01:19:31.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.634+0000 7f7fa314c700 1 -- 172.21.15.22:0/1255999946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c0cbd00 msgr2=0x7f7f9c1624c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.634+0000 7f7fa314c700 1 --2- 172.21.15.22:0/1255999946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f9c0cbd00 0x7f7f9c1624c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c650 tx=0x7f7f8c00c960 comp rx=0 tx=0).stop 2023-07-06T01:19:31.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.634+0000 7f304a91e700 1 -- 172.21.15.22:0/229179646 --> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f30440ce760 con 0x7f3044158160 2023-07-06T01:19:31.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.634+0000 7f7fa16e9700 1 -- 172.21.15.22:0/1255999946 reap_dead start 2023-07-06T01:19:31.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.634+0000 7f7fa314c700 1 -- 172.21.15.22:0/1255999946 shutdown_connections 2023-07-06T01:19:31.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.634+0000 7f7fa314c700 1 -- 172.21.15.22:0/1255999946 >> 172.21.15.22:0/1255999946 conn(0x7f7f9c06b110 msgr2=0x7f7f9c0afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.635+0000 7f7fa314c700 1 -- 172.21.15.22:0/1255999946 shutdown_connections 2023-07-06T01:19:31.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.635+0000 7f0e324cc700 1 -- 172.21.15.22:0/3448000391 shutdown_connections 2023-07-06T01:19:31.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.635+0000 7f0e324cc700 1 -- 172.21.15.22:0/3448000391 wait complete. 2023-07-06T01:19:31.696 INFO:teuthology.orchestra.run.smithi022.stdout:55834574853 2023-07-06T01:19:31.696 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-06T01:19:31.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.635+0000 7f7fa314c700 1 -- 172.21.15.22:0/1255999946 wait complete. 2023-07-06T01:19:31.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.635+0000 7f3040ff9700 1 -- 172.21.15.22:0/229179646 <== osd.4 v2:172.21.15.103:6824/3712184942 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3038011b00 con 0x7f3044158160 2023-07-06T01:19:31.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.636+0000 7f304a91e700 1 -- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] conn(0x7f3044158160 msgr2=0x7f30440c9e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.636+0000 7f304a91e700 1 --2- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] conn(0x7f3044158160 0x7f30440c9e60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.636+0000 7f304a91e700 1 -- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f303006adb0 msgr2=0x7f303006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.636+0000 7f304a91e700 1 --2- 172.21.15.22:0/229179646 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f303006adb0 0x7f303006d260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f303400d2d0 tx=0x7f3034008040 comp rx=0 tx=0).stop 2023-07-06T01:19:31.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.636+0000 7f304a91e700 1 -- 172.21.15.22:0/229179646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30440acdb0 msgr2=0x7f3044157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.636+0000 7f304a91e700 1 --2- 172.21.15.22:0/229179646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30440acdb0 0x7f3044157500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f303c00cbc0 tx=0x7f303c00cf80 comp rx=0 tx=0).stop 2023-07-06T01:19:31.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.636+0000 7f3048ebb700 1 -- 172.21.15.22:0/229179646 reap_dead start 2023-07-06T01:19:31.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.636+0000 7f304a91e700 1 -- 172.21.15.22:0/229179646 shutdown_connections 2023-07-06T01:19:31.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.636+0000 7f304a91e700 1 -- 172.21.15.22:0/229179646 >> 172.21.15.22:0/229179646 conn(0x7f30440b13a0 msgr2=0x7f30440c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.638+0000 7f304a91e700 1 -- 172.21.15.22:0/229179646 shutdown_connections 2023-07-06T01:19:31.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.638+0000 7f304a91e700 1 -- 172.21.15.22:0/229179646 wait complete. 2023-07-06T01:19:31.700 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:19:31.701 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2023-07-06T01:19:31.701 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-06T01:19:31.702 INFO:teuthology.orchestra.run.smithi022.stdout:55834574852 2023-07-06T01:19:31.702 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-06T01:19:31.703 INFO:teuthology.orchestra.run.smithi022.stdout:55834574854 2023-07-06T01:19:31.703 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-06T01:19:31.704 INFO:teuthology.orchestra.run.smithi022.stdout:55834574853 2023-07-06T01:19:31.705 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-06T01:19:31.705 INFO:teuthology.orchestra.run.smithi022.stdout:55834574853 2023-07-06T01:19:31.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-06T01:19:31.706 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2023-07-06T01:19:31.707 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-06T01:19:31.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.959+0000 7f8332362700 1 Processor -- start 2023-07-06T01:19:31.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.960+0000 7f8332362700 1 -- start start 2023-07-06T01:19:31.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.961+0000 7f8332362700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f832c161010 0x7f832c0cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.961+0000 7f8332362700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c165790 con 0x7f832c06d810 2023-07-06T01:19:31.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.961+0000 7f8332362700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c06c9f0 con 0x7f832c06d3a0 2023-07-06T01:19:31.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.961+0000 7f8332362700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c0714d0 con 0x7f832c161010 2023-07-06T01:19:31.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.962+0000 7f832b7fe700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f832c06d3a0 0x7f832c164970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38334/0 (socket says 172.21.15.22:38334) 2023-07-06T01:19:31.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.962+0000 7f83308ff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f832c06d810 0x7f832c165080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36268/0 (socket says 172.21.15.22:36268) 2023-07-06T01:19:31.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.962+0000 7f832b7fe700 1 -- 172.21.15.22:0/291231049 learned_addr learned my addr 172.21.15.22:0/291231049 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:31.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.963+0000 7f832bfff700 1 --2- 172.21.15.22:0/291231049 >> v2:172.21.15.103:3300/0 conn(0x7f832c161010 0x7f832c0cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.963+0000 7f832bfff700 1 -- 172.21.15.22:0/291231049 >> v1:172.21.15.103:6789/0 conn(0x7f832c06d3a0 legacy=0x7f832c164970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.963+0000 7f832bfff700 1 -- 172.21.15.22:0/291231049 >> v1:172.21.15.22:6789/0 conn(0x7f832c06d810 legacy=0x7f832c165080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.963+0000 7f832bfff700 1 -- 172.21.15.22:0/291231049 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0717b0 con 0x7f832c161010 2023-07-06T01:19:31.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.963+0000 7f832bfff700 1 --2- 172.21.15.22:0/291231049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832c161010 0x7f832c0cf4a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8314009840 tx=0x7f8314009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2641598e7fb8793b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.963+0000 7f5de88be700 1 Processor -- start 2023-07-06T01:19:31.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.963+0000 7f90ac0b4700 1 Processor -- start 2023-07-06T01:19:31.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.963+0000 7f5de88be700 1 -- start start 2023-07-06T01:19:31.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.964+0000 7f90ac0b4700 1 -- start start 2023-07-06T01:19:31.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.965+0000 7f5de88be700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5de415c0f0 0x7f5de40cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.965+0000 7f5de88be700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f5de406dab0 0x7f5de4163910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.965+0000 7f5de88be700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de40cb1c0 con 0x7f5de40cf290 2023-07-06T01:19:31.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.965+0000 7f5de88be700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de40cb410 con 0x7f5de406dab0 2023-07-06T01:19:31.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.965+0000 7f5de88be700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de415d4e0 con 0x7f5de415c0f0 2023-07-06T01:19:31.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.965+0000 7f5de259c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5de415c0f0 0x7f5de40cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.965+0000 7f5de2d9d700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f5de406dab0 0x7f5de4163910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.965+0000 7f5de259c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5de415c0f0 0x7f5de40cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56668/0 (socket says 172.21.15.22:56668) 2023-07-06T01:19:31.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.965+0000 7f5de259c700 1 -- 172.21.15.22:0/113863100 learned_addr learned my addr 172.21.15.22:0/113863100 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f90ac0b4700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f90a406be40 0x7f90a406ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f90ac0b4700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a40caf80 con 0x7f90a4158930 2023-07-06T01:19:31.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f90ac0b4700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a406b360 con 0x7f90a40cdf40 2023-07-06T01:19:31.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f90ac0b4700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a40c92e0 con 0x7f90a406be40 2023-07-06T01:19:31.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f90a9e50700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f90a4158930 0x7f90a40ca6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36282/0 (socket says 172.21.15.22:36282) 2023-07-06T01:19:31.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f90a9e50700 1 -- 172.21.15.22:0/2365811972 learned_addr learned my addr 172.21.15.22:0/2365811972 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:31.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.965+0000 7f832affd700 1 -- 172.21.15.22:0/291231049 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314004230 con 0x7f832c161010 2023-07-06T01:19:31.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.965+0000 7f832affd700 1 -- 172.21.15.22:0/291231049 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8314004390 con 0x7f832c161010 2023-07-06T01:19:31.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.965+0000 7f832affd700 1 -- 172.21.15.22:0/291231049 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83140036a0 con 0x7f832c161010 2023-07-06T01:19:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f8332362700 1 -- 172.21.15.22:0/291231049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832c161010 msgr2=0x7f832c0cf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f8332362700 1 --2- 172.21.15.22:0/291231049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832c161010 0x7f832c0cf4a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8314009840 tx=0x7f8314009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027019107 0 0) 0x7f90a40caf80 con 0x7f90a4158930 2023-07-06T01:19:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90900046c0 con 0x7f90a4158930 2023-07-06T01:19:31.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a964f700 1 --2- 172.21.15.22:0/2365811972 >> v2:172.21.15.22:3300/0 conn(0x7f90a406be40 0x7f90a406ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3811845618 0 0) 0x7f90900046c0 con 0x7f90a4158930 2023-07-06T01:19:31.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 >> v2:172.21.15.22:3300/0 conn(0x7f90a406be40 msgr2=0x7f90a406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 --2- 172.21.15.22:0/2365811972 >> v2:172.21.15.22:3300/0 conn(0x7f90a406be40 0x7f90a406ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 >> v1:172.21.15.103:6789/0 conn(0x7f90a40cdf40 legacy=0x7f90a406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a406b640 con 0x7f90a4158930 2023-07-06T01:19:31.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f909c0050e0 con 0x7f90a4158930 2023-07-06T01:19:31.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 >> v1:172.21.15.22:6789/0 conn(0x7f90a4158930 legacy=0x7f90a40ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 --2- 172.21.15.22:0/2365811972 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9090004b10 0x7f9090004f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 --2- 172.21.15.22:0/2365811972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9090005960 0x7f9090005dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 --2- 172.21.15.22:0/2365811972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9090006310 0x7f909000cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40c92e0 con 0x7f9090005960 2023-07-06T01:19:31.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909000d220 con 0x7f9090006310 2023-07-06T01:19:31.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909000d3e0 con 0x7f9090004b10 2023-07-06T01:19:31.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f8332362700 1 -- 172.21.15.22:0/291231049 shutdown_connections 2023-07-06T01:19:31.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f8332362700 1 --2- 172.21.15.22:0/291231049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832c161010 0x7f832c0cf4a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f8332362700 1 -- 172.21.15.22:0/291231049 >> 172.21.15.22:0/291231049 conn(0x7f832c0b13a0 msgr2=0x7f832c0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899988777 0 0) 0x7f5de40cb1c0 con 0x7f5de40cf290 2023-07-06T01:19:31.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dcc0035b0 con 0x7f5de40cf290 2023-07-06T01:19:31.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 <== mon.1 v1:172.21.15.103:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1961681200 0 0) 0x7f5dcc0035b0 con 0x7f5de40cf290 2023-07-06T01:19:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5de415c0f0 msgr2=0x7f5de40cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 --2- 172.21.15.22:0/113863100 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5de415c0f0 0x7f5de40cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 >> v2:172.21.15.103:3300/0 conn(0x7f5de406dab0 msgr2=0x7f5de4163910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 --2- 172.21.15.22:0/113863100 >> v2:172.21.15.103:3300/0 conn(0x7f5de406dab0 0x7f5de4163910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 --> v1:172.21.15.103:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de406caf0 con 0x7f5de40cf290 2023-07-06T01:19:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 <== mon.1 v1:172.21.15.103:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f5dd8001670 con 0x7f5de40cf290 2023-07-06T01:19:31.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 >> v1:172.21.15.103:6789/0 conn(0x7f5de40cf290 legacy=0x7f5de406d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 --2- 172.21.15.22:0/113863100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dcc003a00 0x7f5dcc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 --2- 172.21.15.22:0/113863100 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5dcc004850 0x7f5dcc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 --2- 172.21.15.22:0/113863100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dcc005200 0x7f5dcc009a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de415d4e0 con 0x7f5dcc005200 2023-07-06T01:19:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc009f40 con 0x7f5dcc003a00 2023-07-06T01:19:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.966+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc00a100 con 0x7f5dcc004850 2023-07-06T01:19:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f5de1d9b700 1 --2- 172.21.15.22:0/113863100 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5dcc004850 0x7f5dcc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f5de2d9d700 1 --2- 172.21.15.22:0/113863100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dcc005200 0x7f5dcc009a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f5de1d9b700 1 -- 172.21.15.22:0/113863100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dcc003a00 msgr2=0x7f5dcc003e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T01:19:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f5de1d9b700 1 --2- 172.21.15.22:0/113863100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5dcc003a00 0x7f5dcc003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f5de1d9b700 1 -- 172.21.15.22:0/113863100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dcc005200 msgr2=0x7f5dcc009a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f5de1d9b700 1 --2- 172.21.15.22:0/113863100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dcc005200 0x7f5dcc009a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f5de1d9b700 1 -- 172.21.15.22:0/113863100 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc0035b0 con 0x7f5dcc004850 2023-07-06T01:19:31.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f8332362700 1 -- 172.21.15.22:0/291231049 shutdown_connections 2023-07-06T01:19:31.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f8332362700 1 -- 172.21.15.22:0/291231049 wait complete. 2023-07-06T01:19:31.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.967+0000 7f90aa651700 1 --2- 172.21.15.22:0/2365811972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9090006310 0x7f909000cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.968+0000 7f90a9e50700 1 --2- 172.21.15.22:0/2365811972 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9090004b10 0x7f9090004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.968+0000 7f5de1d9b700 1 --2- 172.21.15.22:0/113863100 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5dcc004850 0x7f5dcc004cc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8007fb0 tx=0x7f5dcc005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfdde0c30029935d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.968+0000 7f90aa651700 1 -- 172.21.15.22:0/2365811972 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9090004b10 msgr2=0x7f9090004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.968+0000 7f90aa651700 1 --2- 172.21.15.22:0/2365811972 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9090004b10 0x7f9090004f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.968+0000 7f90aa651700 1 -- 172.21.15.22:0/2365811972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9090005960 msgr2=0x7f9090005dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T01:19:31.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.968+0000 7f90aa651700 1 --2- 172.21.15.22:0/2365811972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9090005960 0x7f9090005dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.968+0000 7f90aa651700 1 -- 172.21.15.22:0/2365811972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a406b640 con 0x7f9090006310 2023-07-06T01:19:31.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.968+0000 7f90aa651700 1 -- 172.21.15.22:0/2365811972 reap_dead start 2023-07-06T01:19:31.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.968+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800cc50 con 0x7f5dcc004850 2023-07-06T01:19:31.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.968+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd800cdb0 con 0x7f5dcc004850 2023-07-06T01:19:31.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.968+0000 7f5de159a700 1 -- 172.21.15.22:0/113863100 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8012620 con 0x7f5dcc004850 2023-07-06T01:19:31.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.968+0000 7f90aa651700 1 --2- 172.21.15.22:0/2365811972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9090006310 0x7f909000cb20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f90a000ea60 tx=0x7f9090006990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd925895afa52d01 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7fd5476c6700 1 Processor -- start 2023-07-06T01:19:31.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7f9111070700 1 Processor -- start 2023-07-06T01:19:31.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7f9111070700 1 -- start start 2023-07-06T01:19:31.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7f8332362700 1 Processor -- start 2023-07-06T01:19:31.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7fd5476c6700 1 -- start start 2023-07-06T01:19:31.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7f5de88be700 1 -- 172.21.15.22:0/113863100 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5dcc004850 msgr2=0x7f5dcc004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7f5de88be700 1 --2- 172.21.15.22:0/113863100 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5dcc004850 0x7f5dcc004cc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8007fb0 tx=0x7f5dcc005880 comp rx=0 tx=0).stop 2023-07-06T01:19:31.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7f5de2d9d700 1 -- 172.21.15.22:0/113863100 reap_dead start 2023-07-06T01:19:31.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f5de88be700 1 -- 172.21.15.22:0/113863100 shutdown_connections 2023-07-06T01:19:31.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f5de88be700 1 -- 172.21.15.22:0/113863100 >> 172.21.15.22:0/113863100 conn(0x7f5de40b13a0 msgr2=0x7f5de40ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7f9f2757a700 1 Processor -- start 2023-07-06T01:19:31.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0013070 con 0x7f9090006310 2023-07-06T01:19:31.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a000ee90 con 0x7f9090006310 2023-07-06T01:19:31.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7f90a8e4e700 1 -- 172.21.15.22:0/2365811972 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a000e550 con 0x7f9090006310 2023-07-06T01:19:31.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f90ac0b4700 1 -- 172.21.15.22:0/2365811972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9090006310 msgr2=0x7f909000cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f90ac0b4700 1 --2- 172.21.15.22:0/2365811972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9090006310 0x7f909000cb20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f90a000ea60 tx=0x7f9090006990 comp rx=0 tx=0).stop 2023-07-06T01:19:31.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f5de88be700 1 -- 172.21.15.22:0/113863100 shutdown_connections 2023-07-06T01:19:31.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f90ac0b4700 1 -- 172.21.15.22:0/2365811972 shutdown_connections 2023-07-06T01:19:31.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f90ac0b4700 1 --2- 172.21.15.22:0/2365811972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9090006310 0x7f909000cb20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f90ac0b4700 1 -- 172.21.15.22:0/2365811972 >> 172.21.15.22:0/2365811972 conn(0x7f90a406ac40 msgr2=0x7f90a40cb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:31.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7fee9ff90700 1 Processor -- start 2023-07-06T01:19:31.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7fee9ff90700 1 -- start start 2023-07-06T01:19:31.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f9111070700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f910c0ada90 0x7f910c0adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f9111070700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910c1609d0 con 0x7f910c15ca60 2023-07-06T01:19:31.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f9111070700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910c160cb0 con 0x7f910c06bda0 2023-07-06T01:19:31.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f9111070700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0ae440 con 0x7f910c0ada90 2023-07-06T01:19:31.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f90ac0b4700 1 -- 172.21.15.22:0/2365811972 shutdown_connections 2023-07-06T01:19:31.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.969+0000 7f8332362700 1 -- start start 2023-07-06T01:19:31.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f8332362700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832c06d3a0 0x7f832c0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f8332362700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c06d810 0x7f832c0c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f8332362700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f832c161010 0x7f832c15ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f8332362700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0c5ce0 con 0x7f832c06d810 2023-07-06T01:19:31.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f8332362700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0b3e80 con 0x7f832c06d3a0 2023-07-06T01:19:31.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f8332362700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0b4160 con 0x7f832c161010 2023-07-06T01:19:31.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f83308ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f832c161010 0x7f832c15ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f832bfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832c06d3a0 0x7f832c0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f910a59c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f910c0ada90 0x7f910c0adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f910a59c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f910c0ada90 0x7f910c0adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56692/0 (socket says 172.21.15.22:56692) 2023-07-06T01:19:31.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f910a59c700 1 -- 172.21.15.22:0/1678101800 learned_addr learned my addr 172.21.15.22:0/1678101800 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7fd5476c6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd540161010 0x7fd54006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:31.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7fd5476c6700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5400cc880 con 0x7fd5400c2290 2023-07-06T01:19:31.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7fd5476c6700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd540071a00 con 0x7fd5400cd420 2023-07-06T01:19:31.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7fd5476c6700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54006cd40 con 0x7fd540161010 2023-07-06T01:19:31.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f83308ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f832c161010 0x7f832c15ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56698/0 (socket says 172.21.15.22:56698) 2023-07-06T01:19:31.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f83308ff700 1 -- 172.21.15.22:0/170103422 learned_addr learned my addr 172.21.15.22:0/170103422 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:31.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f60a31a8700 1 Processor -- start 2023-07-06T01:19:31.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f832b7fe700 1 --2- 172.21.15.22:0/170103422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c06d810 0x7f832c0c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:31.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f83308ff700 1 -- 172.21.15.22:0/170103422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832c06d3a0 msgr2=0x7f832c0b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f83308ff700 1 --2- 172.21.15.22:0/170103422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832c06d3a0 0x7f832c0b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f83308ff700 1 -- 172.21.15.22:0/170103422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c06d810 msgr2=0x7f832c0c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:31.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f83308ff700 1 --2- 172.21.15.22:0/170103422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c06d810 0x7f832c0c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:31.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f83308ff700 1 -- 172.21.15.22:0/170103422 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83140094a0 con 0x7f832c161010 2023-07-06T01:19:31.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f910a59c700 1 -- 172.21.15.22:0/1678101800 >> v1:172.21.15.103:6789/0 conn(0x7f910c06bda0 legacy=0x7f910c06c220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T01:19:31.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f910a59c700 1 -- 172.21.15.22:0/1678101800 >> v1:172.21.15.22:6789/0 conn(0x7f910c15ca60 legacy=0x7f910c0cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:31.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f910a59c700 1 -- 172.21.15.22:0/1678101800 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c160f90 con 0x7f910c0ada90 2023-07-06T01:19:31.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f910a59c700 1 --2- 172.21.15.22:0/1678101800 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f910c0ada90 0x7f910c0adf00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9100009920 tx=0x7f9100009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a971738188e962ee server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:31.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7fd545c63700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fd5400cd420 0x7fd540164b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38378/0 (socket says 172.21.15.22:38378) 2023-07-06T01:19:31.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7fd545c63700 1 -- 172.21.15.22:0/1586803592 learned_addr learned my addr 172.21.15.22:0/1586803592 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:31.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9109d9b700 1 -- 172.21.15.22:0/1678101800 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100004230 con 0x7f910c0ada90 2023-07-06T01:19:32.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583133768 0 0) 0x7fd5400cc880 con 0x7fd5400c2290 2023-07-06T01:19:32.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd52c0035b0 con 0x7fd5400c2290 2023-07-06T01:19:32.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9109d9b700 1 -- 172.21.15.22:0/1678101800 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9100004390 con 0x7f910c0ada90 2023-07-06T01:19:32.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9109d9b700 1 -- 172.21.15.22:0/1678101800 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100009e70 con 0x7f910c0ada90 2023-07-06T01:19:32.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046427328 0 0) 0x7fd540071a00 con 0x7fd5400cd420 2023-07-06T01:19:32.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5400cc880 con 0x7fd5400cd420 2023-07-06T01:19:32.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4257629577 0 0) 0x7fd52c0035b0 con 0x7fd5400c2290 2023-07-06T01:19:32.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f60a31a8700 1 -- start start 2023-07-06T01:19:32.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 >> v2:172.21.15.22:3300/0 conn(0x7fd540161010 msgr2=0x7fd54006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:32.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 --2- 172.21.15.22:0/1586803592 >> v2:172.21.15.22:3300/0 conn(0x7fd540161010 0x7fd54006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 >> v1:172.21.15.103:6789/0 conn(0x7fd5400cd420 legacy=0x7fd540164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54006d020 con 0x7fd5400c2290 2023-07-06T01:19:32.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7fd530001670 con 0x7fd5400c2290 2023-07-06T01:19:32.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f60a31a8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f609c0d0a90 0x7f609c0d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f60a31a8700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609c0c66f0 con 0x7f609c160e50 2023-07-06T01:19:32.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f60a31a8700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609c0afc70 con 0x7f609c0b0d10 2023-07-06T01:19:32.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f60a31a8700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609c0d1470 con 0x7f609c0d0a90 2023-07-06T01:19:32.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f90ac0b4700 1 -- 172.21.15.22:0/2365811972 wait complete. 2023-07-06T01:19:32.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 >> v1:172.21.15.22:6789/0 conn(0x7fd5400c2290 legacy=0x7fd5400ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 --2- 172.21.15.22:0/1586803592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd52c003b90 0x7fd52c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 --2- 172.21.15.22:0/1586803592 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd52c004850 0x7fd52c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 --2- 172.21.15.22:0/1586803592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd52c005320 0x7fd52c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54006cd40 con 0x7fd52c005320 2023-07-06T01:19:32.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c00c140 con 0x7fd52c003b90 2023-07-06T01:19:32.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c00c300 con 0x7fd52c004850 2023-07-06T01:19:32.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7fd544c61700 1 --2- 172.21.15.22:0/1586803592 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd52c004850 0x7fd52c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f5de88be700 1 -- 172.21.15.22:0/113863100 wait complete. 2023-07-06T01:19:32.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f5de88be700 1 Processor -- start 2023-07-06T01:19:32.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7fee9ff90700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 0x7fee980cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7fee9ff90700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fee9806d810 0x7fee98165080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7fee9ff90700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9806c9f0 con 0x7fee9806d3a0 2023-07-06T01:19:32.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7fee9ff90700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee980714d0 con 0x7fee9806d810 2023-07-06T01:19:32.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7fee9ff90700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee980717b0 con 0x7fee98161010 2023-07-06T01:19:32.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7fee9dd2c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 0x7fee980cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7fee9dd2c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 0x7fee980cf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56710/0 (socket says 172.21.15.22:56710) 2023-07-06T01:19:32.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fee9dd2c700 1 -- 172.21.15.22:0/285354897 learned_addr learned my addr 172.21.15.22:0/285354897 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:32.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fee9dd2c700 1 -- 172.21.15.22:0/285354897 >> v2:172.21.15.22:3300/0 conn(0x7fee9806d810 msgr2=0x7fee98165080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:32.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fee9dd2c700 1 --2- 172.21.15.22:0/285354897 >> v2:172.21.15.22:3300/0 conn(0x7fee9806d810 0x7fee98165080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fee9dd2c700 1 -- 172.21.15.22:0/285354897 >> v1:172.21.15.22:6789/0 conn(0x7fee9806d3a0 legacy=0x7fee98164970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T01:19:32.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fee9dd2c700 1 -- 172.21.15.22:0/285354897 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee98071a90 con 0x7fee98161010 2023-07-06T01:19:32.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fee9dd2c700 1 --2- 172.21.15.22:0/285354897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 0x7fee980cf4a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fee940098f0 tx=0x7fee94009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8401e0cb1e952ea1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7fee9cd2a700 1 -- 172.21.15.22:0/285354897 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94004030 con 0x7fee98161010 2023-07-06T01:19:32.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.970+0000 7f9f2757a700 1 -- start start 2023-07-06T01:19:32.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.971+0000 7f9f2757a700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9f200c2290 0x7f9f2006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f2757a700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f200cc880 con 0x7f9f200cd420 2023-07-06T01:19:32.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f2757a700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f20071a00 con 0x7f9f20161010 2023-07-06T01:19:32.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f2757a700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2006cd40 con 0x7f9f200c2290 2023-07-06T01:19:32.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f25316700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f9f20161010 0x7f9f200ccd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38384/0 (socket says 172.21.15.22:38384) 2023-07-06T01:19:32.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f25316700 1 -- 172.21.15.22:0/3392155317 learned_addr learned my addr 172.21.15.22:0/3392155317 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:32.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955883552 0 0) 0x7f9f20071a00 con 0x7f9f20161010 2023-07-06T01:19:32.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f0c0035b0 con 0x7f9f20161010 2023-07-06T01:19:32.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 <== mon.1 v1:172.21.15.103:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3474631669 0 0) 0x7f9f0c0035b0 con 0x7f9f20161010 2023-07-06T01:19:32.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 >> v2:172.21.15.103:3300/0 conn(0x7f9f200c2290 msgr2=0x7f9f2006f910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T01:19:32.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f17fff700 1 --2- 172.21.15.22:0/3392155317 >> v2:172.21.15.103:3300/0 conn(0x7f9f200c2290 0x7f9f2006f910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 >> v1:172.21.15.22:6789/0 conn(0x7f9f200cd420 legacy=0x7f9f20164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:32.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 --> v1:172.21.15.103:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2006d020 con 0x7f9f20161010 2023-07-06T01:19:32.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 <== mon.1 v1:172.21.15.103:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f9f10001670 con 0x7f9f20161010 2023-07-06T01:19:32.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.972+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 >> v1:172.21.15.103:6789/0 conn(0x7f9f20161010 legacy=0x7f9f200ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f9f17fff700 1 --2- 172.21.15.22:0/3392155317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f0c003a00 0x7f9f0c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f9f17fff700 1 --2- 172.21.15.22:0/3392155317 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f0c004850 0x7f9f0c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f9f17fff700 1 --2- 172.21.15.22:0/3392155317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f0c005320 0x7f9f0c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2006cd40 con 0x7f9f0c005320 2023-07-06T01:19:32.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c00c1d0 con 0x7f9f0c003a00 2023-07-06T01:19:32.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c00c390 con 0x7f9f0c004850 2023-07-06T01:19:32.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f83308ff700 1 --2- 172.21.15.22:0/170103422 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f832c161010 0x7f832c15ef80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8324002000 tx=0x7f832400a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7fd545462700 1 --2- 172.21.15.22:0/1586803592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd52c003b90 0x7fd52c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f60a1745700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f609c0b0d10 0x7f609c0b1110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38398/0 (socket says 172.21.15.22:38398) 2023-07-06T01:19:32.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f60a1745700 1 -- 172.21.15.22:0/1710564096 learned_addr learned my addr 172.21.15.22:0/1710564096 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:32.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f8328ff9700 1 -- 172.21.15.22:0/170103422 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832400ad40 con 0x7f832c161010 2023-07-06T01:19:32.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7fee9cd2a700 1 -- 172.21.15.22:0/285354897 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee94004190 con 0x7fee98161010 2023-07-06T01:19:32.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7fee9cd2a700 1 -- 172.21.15.22:0/285354897 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94004520 con 0x7fee98161010 2023-07-06T01:19:32.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f609bfff700 1 --2- 172.21.15.22:0/1710564096 >> v2:172.21.15.22:3300/0 conn(0x7f609c0d0a90 0x7f609c0d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f609b7fe700 1 -- 172.21.15.22:0/1710564096 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223816074 0 0) 0x7f609c0afc70 con 0x7f609c0b0d10 2023-07-06T01:19:32.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f609b7fe700 1 -- 172.21.15.22:0/1710564096 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60840035b0 con 0x7f609c0b0d10 2023-07-06T01:19:32.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f609bfff700 1 -- 172.21.15.22:0/1710564096 >> v1:172.21.15.103:6789/0 conn(0x7f609c0b0d10 legacy=0x7f609c0b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f609bfff700 1 -- 172.21.15.22:0/1710564096 >> v1:172.21.15.22:6789/0 conn(0x7f609c160e50 legacy=0x7f609c0cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:32.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f609bfff700 1 -- 172.21.15.22:0/1710564096 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c0aff50 con 0x7f609c0d0a90 2023-07-06T01:19:32.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f609bfff700 1 --2- 172.21.15.22:0/1710564096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f609c0d0a90 0x7f609c0d0f30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6090009840 tx=0x7f6090009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cec191d2d3b940 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f609b7fe700 1 -- 172.21.15.22:0/1710564096 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090004290 con 0x7f609c0d0a90 2023-07-06T01:19:32.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f609b7fe700 1 -- 172.21.15.22:0/1710564096 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60900043f0 con 0x7f609c0d0a90 2023-07-06T01:19:32.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f609b7fe700 1 -- 172.21.15.22:0/1710564096 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60900038a0 con 0x7f609c0d0a90 2023-07-06T01:19:32.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f90ac0b4700 1 Processor -- start 2023-07-06T01:19:32.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f90ac0b4700 1 -- start start 2023-07-06T01:19:32.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f90ac0b4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9090006310 0x7f90a4153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f90ac0b4700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90a40b18a0 0x7f90a4142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f90ac0b4700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90a4143400 0x7f90a40c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f90ac0b4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a406ba20 con 0x7f9090006310 2023-07-06T01:19:32.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f90ac0b4700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40711a0 con 0x7f90a4143400 2023-07-06T01:19:32.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f90ac0b4700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4143870 con 0x7f90a40b18a0 2023-07-06T01:19:32.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f90a9e50700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9090006310 0x7f90a4153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f90a9e50700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9090006310 0x7f90a4153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45604/0 (socket says 172.21.15.22:45604) 2023-07-06T01:19:32.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f90a9e50700 1 -- 172.21.15.22:0/46916284 learned_addr learned my addr 172.21.15.22:0/46916284 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:32.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f8332362700 1 -- 172.21.15.22:0/170103422 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c0b4440 con 0x7f832c161010 2023-07-06T01:19:32.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f8332362700 1 -- 172.21.15.22:0/170103422 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c0b3ab0 con 0x7f832c161010 2023-07-06T01:19:32.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f8328ff9700 1 -- 172.21.15.22:0/170103422 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83240070d0 con 0x7f832c161010 2023-07-06T01:19:32.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f8328ff9700 1 -- 172.21.15.22:0/170103422 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832400aea0 con 0x7f832c161010 2023-07-06T01:19:32.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f9111070700 1 -- 172.21.15.22:0/1678101800 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f910c0ada90 msgr2=0x7f910c0adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f9111070700 1 --2- 172.21.15.22:0/1678101800 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f910c0ada90 0x7f910c0adf00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9100009920 tx=0x7f9100009c30 comp rx=0 tx=0).stop 2023-07-06T01:19:32.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f9111070700 1 -- 172.21.15.22:0/1678101800 shutdown_connections 2023-07-06T01:19:32.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f9111070700 1 --2- 172.21.15.22:0/1678101800 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f910c0ada90 0x7f910c0adf00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f9111070700 1 -- 172.21.15.22:0/1678101800 >> 172.21.15.22:0/1678101800 conn(0x7f910c0b0710 msgr2=0x7f910c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:32.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7fd545462700 1 -- 172.21.15.22:0/1586803592 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd52c004850 msgr2=0x7fd52c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:32.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7fd545462700 1 --2- 172.21.15.22:0/1586803592 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd52c004850 0x7fd52c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7fd545462700 1 -- 172.21.15.22:0/1586803592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd52c005320 msgr2=0x7fd52c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:32.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7fd545462700 1 --2- 172.21.15.22:0/1586803592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd52c005320 0x7fd52c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7fd545c63700 1 -- 172.21.15.22:0/1586803592 reap_dead start 2023-07-06T01:19:32.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7fd545462700 1 -- 172.21.15.22:0/1586803592 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54006d020 con 0x7fd52c003b90 2023-07-06T01:19:32.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7fd545462700 1 --2- 172.21.15.22:0/1586803592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd52c003b90 0x7fd52c003f80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009910 tx=0x7fd52c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14a76d0e253406e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7fd544c61700 1 --2- 172.21.15.22:0/1586803592 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd52c004850 0x7fd52c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:32.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f5de88be700 1 -- start start 2023-07-06T01:19:32.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f5de88be700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dcc004850 0x7f5de4154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f5de88be700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5de40c6240 0x7f5de41ed780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f5de88be700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5de40c66b0 0x7f5de41edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f5de88be700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de4145500 con 0x7f5dcc004850 2023-07-06T01:19:32.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f5de88be700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de41ee200 con 0x7f5de40c6240 2023-07-06T01:19:32.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f5de88be700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de4157a40 con 0x7f5de40c66b0 2023-07-06T01:19:32.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f5de259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dcc004850 0x7f5de4154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f5de259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dcc004850 0x7f5de4154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45598/0 (socket says 172.21.15.22:45598) 2023-07-06T01:19:32.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f5de259c700 1 -- 172.21.15.22:0/1778102195 learned_addr learned my addr 172.21.15.22:0/1778102195 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:32.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f5de1d9b700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5de40c6240 0x7f5de41ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f5de2d9d700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5de40c66b0 0x7f5de41edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7fee9ff90700 1 -- 172.21.15.22:0/285354897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 msgr2=0x7fee980cf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7fee9ff90700 1 --2- 172.21.15.22:0/285354897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 0x7fee980cf4a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fee940098f0 tx=0x7fee94009c00 comp rx=0 tx=0).stop 2023-07-06T01:19:32.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.973+0000 7f9f24b15700 1 --2- 172.21.15.22:0/3392155317 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f0c004850 0x7f9f0c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f9f24b15700 1 -- 172.21.15.22:0/3392155317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f0c003a00 msgr2=0x7f9f0c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:32.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f9f24b15700 1 --2- 172.21.15.22:0/3392155317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f0c003a00 0x7f9f0c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f9f24b15700 1 -- 172.21.15.22:0/3392155317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f0c005320 msgr2=0x7f9f0c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:32.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f9f24b15700 1 --2- 172.21.15.22:0/3392155317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f0c005320 0x7f9f0c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f9f24b15700 1 -- 172.21.15.22:0/3392155317 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2006d020 con 0x7f9f0c004850 2023-07-06T01:19:32.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.974+0000 7f9f25b17700 1 -- 172.21.15.22:0/3392155317 reap_dead start 2023-07-06T01:19:32.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f9111070700 1 -- 172.21.15.22:0/1678101800 shutdown_connections 2023-07-06T01:19:32.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f9f24b15700 1 --2- 172.21.15.22:0/3392155317 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f0c004850 0x7f9f0c004cc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009e30 tx=0x7f9f0c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e9474d10eb1bc8d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c011620 con 0x7fd52c003b90 2023-07-06T01:19:32.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c011780 con 0x7fd52c003b90 2023-07-06T01:19:32.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7fd537fff700 1 -- 172.21.15.22:0/1586803592 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c011b10 con 0x7fd52c003b90 2023-07-06T01:19:32.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f90aa651700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90a4143400 0x7f90a40c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f90a964f700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90a40b18a0 0x7f90a4142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f8332362700 1 -- 172.21.15.22:0/170103422 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83100052f0 con 0x7f832c161010 2023-07-06T01:19:32.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f5de259c700 1 -- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5de40c66b0 msgr2=0x7f5de41edcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f5de259c700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5de40c66b0 0x7f5de41edcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f5de259c700 1 -- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5de40c6240 msgr2=0x7f5de41ed780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f5de259c700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5de40c6240 0x7f5de41ed780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f5de259c700 1 -- 172.21.15.22:0/1778102195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de413f110 con 0x7f5dcc004850 2023-07-06T01:19:32.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7f5de2d9d700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5de40c66b0 0x7f5de41edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:32.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f5de259c700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dcc004850 0x7f5de4154950 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400fae0 tx=0x7f5dd400fdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7fee9ff90700 1 -- 172.21.15.22:0/285354897 shutdown_connections 2023-07-06T01:19:32.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7fee9ff90700 1 --2- 172.21.15.22:0/285354897 >> v2:172.21.15.22:3300/0 conn(0x7fee9806d810 0x7fee98165080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7fee9ff90700 1 --2- 172.21.15.22:0/285354897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 0x7fee980cf4a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.975+0000 7fee9ff90700 1 -- 172.21.15.22:0/285354897 >> 172.21.15.22:0/285354897 conn(0x7fee980b13a0 msgr2=0x7fee980ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:32.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f60a31a8700 1 -- 172.21.15.22:0/1710564096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f609c0d0a90 msgr2=0x7f609c0d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f60a31a8700 1 --2- 172.21.15.22:0/1710564096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f609c0d0a90 0x7f609c0d0f30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6090009840 tx=0x7f6090009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:32.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f60a31a8700 1 -- 172.21.15.22:0/1710564096 shutdown_connections 2023-07-06T01:19:32.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f60a31a8700 1 --2- 172.21.15.22:0/1710564096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f609c0d0a90 0x7f609c0d0f30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f60a31a8700 1 -- 172.21.15.22:0/1710564096 >> 172.21.15.22:0/1710564096 conn(0x7f609c09c2e0 msgr2=0x7f609c0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:32.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f90a9e50700 1 -- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90a40b18a0 msgr2=0x7f90a4142ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f90a9e50700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90a40b18a0 0x7f90a4142ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f90a9e50700 1 -- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90a4143400 msgr2=0x7f90a40c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f90a9e50700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90a4143400 0x7f90a40c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f90a9e50700 1 -- 172.21.15.22:0/46916284 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a4070590 con 0x7f9090006310 2023-07-06T01:19:32.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f90a9e50700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9090006310 0x7f90a4153d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f909c007a90 tx=0x7f909c007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f8328ff9700 1 -- 172.21.15.22:0/170103422 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8324007480 con 0x7f832c161010 2023-07-06T01:19:32.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f8328ff9700 1 --2- 172.21.15.22:0/170103422 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f830c06ace0 0x7f830c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f8328ff9700 1 -- 172.21.15.22:0/170103422 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8324089200 con 0x7f832c161010 2023-07-06T01:19:32.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f9111070700 1 -- 172.21.15.22:0/1678101800 wait complete. 2023-07-06T01:19:32.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7fd5476c6700 1 -- 172.21.15.22:0/1586803592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd52c003b90 msgr2=0x7fd52c003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7fd5476c6700 1 --2- 172.21.15.22:0/1586803592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd52c003b90 0x7fd52c003f80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009910 tx=0x7fd52c005880 comp rx=0 tx=0).stop 2023-07-06T01:19:32.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7fee9ff90700 1 -- 172.21.15.22:0/285354897 shutdown_connections 2023-07-06T01:19:32.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7fee9ff90700 1 -- 172.21.15.22:0/285354897 wait complete. 2023-07-06T01:19:32.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f180098d0 con 0x7f9f0c004850 2023-07-06T01:19:32.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1800baa0 con 0x7f9f0c004850 2023-07-06T01:19:32.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.976+0000 7f9f17fff700 1 -- 172.21.15.22:0/3392155317 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800be50 con 0x7f9f0c004850 2023-07-06T01:19:32.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f909a7fc700 1 -- 172.21.15.22:0/46916284 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c0072d0 con 0x7f9090006310 2023-07-06T01:19:32.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f909a7fc700 1 -- 172.21.15.22:0/46916284 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c010850 con 0x7f9090006310 2023-07-06T01:19:32.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f909a7fc700 1 -- 172.21.15.22:0/46916284 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c010c00 con 0x7f9090006310 2023-07-06T01:19:32.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f832bfff700 1 --2- 172.21.15.22:0/170103422 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f830c06ace0 0x7f830c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f9f2757a700 1 -- 172.21.15.22:0/3392155317 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f0c004850 msgr2=0x7f9f0c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f9f2757a700 1 --2- 172.21.15.22:0/3392155317 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f0c004850 0x7f9f0c004cc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009e30 tx=0x7f9f0c005880 comp rx=0 tx=0).stop 2023-07-06T01:19:32.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f60a31a8700 1 -- 172.21.15.22:0/1710564096 shutdown_connections 2023-07-06T01:19:32.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f60a31a8700 1 -- 172.21.15.22:0/1710564096 wait complete. 2023-07-06T01:19:32.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f5dd2ffd700 1 -- 172.21.15.22:0/1778102195 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400dbd0 con 0x7f5dcc004850 2023-07-06T01:19:32.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f5dd2ffd700 1 -- 172.21.15.22:0/1778102195 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd400dd30 con 0x7f5dcc004850 2023-07-06T01:19:32.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f5de88be700 1 -- 172.21.15.22:0/1778102195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de4157480 con 0x7f5dcc004850 2023-07-06T01:19:32.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f5de88be700 1 -- 172.21.15.22:0/1778102195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de40b24c0 con 0x7f5dcc004850 2023-07-06T01:19:32.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7fee9ff90700 1 Processor -- start 2023-07-06T01:19:32.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7fee9ff90700 1 -- start start 2023-07-06T01:19:32.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7f832bfff700 1 --2- 172.21.15.22:0/170103422 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f830c06ace0 0x7f830c06d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8314000c00 tx=0x7f8314008000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7f9111070700 1 Processor -- start 2023-07-06T01:19:32.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f5dd2ffd700 1 -- 172.21.15.22:0/1778102195 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400dea0 con 0x7f5dcc004850 2023-07-06T01:19:32.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7fd5476c6700 1 -- 172.21.15.22:0/1586803592 shutdown_connections 2023-07-06T01:19:32.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7fd5476c6700 1 --2- 172.21.15.22:0/1586803592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd52c003b90 0x7fd52c003f80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7fd5476c6700 1 -- 172.21.15.22:0/1586803592 >> 172.21.15.22:0/1586803592 conn(0x7fd5400b0710 msgr2=0x7fd5400b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:32.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7f9f2757a700 1 -- 172.21.15.22:0/3392155317 shutdown_connections 2023-07-06T01:19:32.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7f9f2757a700 1 --2- 172.21.15.22:0/3392155317 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f0c004850 0x7f9f0c004cc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7f9f2757a700 1 -- 172.21.15.22:0/3392155317 >> 172.21.15.22:0/3392155317 conn(0x7f9f200b0710 msgr2=0x7f9f200b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:32.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7f9111070700 1 -- start start 2023-07-06T01:19:32.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7f9f2757a700 1 -- 172.21.15.22:0/3392155317 shutdown_connections 2023-07-06T01:19:32.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7f9f2757a700 1 -- 172.21.15.22:0/3392155317 wait complete. 2023-07-06T01:19:32.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7f60a31a8700 1 Processor -- start 2023-07-06T01:19:32.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7f60a31a8700 1 -- start start 2023-07-06T01:19:32.041 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7fee9ff90700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee9806d3a0 0x7fee981571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7fee9ff90700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee9806d810 0x7fee980af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7fee9ff90700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 0x7fee980afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7fee9ff90700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee980c4b60 con 0x7fee9806d810 2023-07-06T01:19:32.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7fee9ff90700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee981576e0 con 0x7fee9806d3a0 2023-07-06T01:19:32.042 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7fee9ff90700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee980c3f50 con 0x7fee98161010 2023-07-06T01:19:32.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f90ac0b4700 1 -- 172.21.15.22:0/46916284 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a4070870 con 0x7f9090006310 2023-07-06T01:19:32.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.977+0000 7f90ac0b4700 1 -- 172.21.15.22:0/46916284 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a4070e30 con 0x7f9090006310 2023-07-06T01:19:32.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.978+0000 7f90ac0b4700 1 -- 172.21.15.22:0/46916284 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a40744b0 con 0x7f9090006310 2023-07-06T01:19:32.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7fee9e52d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 0x7fee980afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.043 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7fee9e52d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 0x7fee980afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56750/0 (socket says 172.21.15.22:56750) 2023-07-06T01:19:32.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7fee9e52d700 1 -- 172.21.15.22:0/712024624 learned_addr learned my addr 172.21.15.22:0/712024624 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:32.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7fee9e52d700 1 -- 172.21.15.22:0/712024624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee9806d3a0 msgr2=0x7fee981571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7fee9e52d700 1 --2- 172.21.15.22:0/712024624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee9806d3a0 0x7fee981571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7fee9e52d700 1 -- 172.21.15.22:0/712024624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee9806d810 msgr2=0x7fee980af6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T01:19:32.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7fee9e52d700 1 --2- 172.21.15.22:0/712024624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee9806d810 0x7fee980af6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.044 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7fee9e52d700 1 -- 172.21.15.22:0/712024624 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee94009550 con 0x7fee98161010 2023-07-06T01:19:32.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f60a31a8700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f609c0b0d10 0x7f609c15e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f60a31a8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f609c0d0a90 0x7f609c0cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f60a31a8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f609c160e50 0x7f609c0cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.045 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f60a31a8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c15ada0 con 0x7f609c0d0a90 2023-07-06T01:19:32.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f60a31a8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c15eba0 con 0x7f609c160e50 2023-07-06T01:19:32.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f60a31a8700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609c15ee80 con 0x7f609c0b0d10 2023-07-06T01:19:32.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f60a0f44700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f609c0b0d10 0x7f609c15e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f60a0f44700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f609c0b0d10 0x7f609c15e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56760/0 (socket says 172.21.15.22:56760) 2023-07-06T01:19:32.046 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f60a0f44700 1 -- 172.21.15.22:0/820607425 learned_addr learned my addr 172.21.15.22:0/820607425 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:32.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7fee9e52d700 1 --2- 172.21.15.22:0/712024624 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 0x7fee980afbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fee8800b440 tx=0x7fee8800b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.047 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7f5de88be700 1 -- 172.21.15.22:0/1778102195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc80052f0 con 0x7f5dcc004850 2023-07-06T01:19:32.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7f9111070700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f910c06bda0 0x7f910c1ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7f9111070700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f910c0ada90 0x7f910c0c3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7f9111070700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f910c15ca60 0x7f910c0c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.979+0000 7f9111070700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0c4e80 con 0x7f910c0ada90 2023-07-06T01:19:32.048 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f9111070700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0c4270 con 0x7f910c15ca60 2023-07-06T01:19:32.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f9111070700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0c4550 con 0x7f910c06bda0 2023-07-06T01:19:32.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f910a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f910c0ada90 0x7f910c0c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f910ad9d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f910c06bda0 0x7f910c1ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f910a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f910c0ada90 0x7f910c0c3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45616/0 (socket says 172.21.15.22:45616) 2023-07-06T01:19:32.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f910a59c700 1 -- 172.21.15.22:0/2624115293 learned_addr learned my addr 172.21.15.22:0/2624115293 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:32.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f910a59c700 1 -- 172.21.15.22:0/2624115293 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f910c06bda0 msgr2=0x7f910c1ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f910a59c700 1 --2- 172.21.15.22:0/2624115293 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f910c06bda0 0x7f910c1ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.049 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f910a59c700 1 -- 172.21.15.22:0/2624115293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f910c15ca60 msgr2=0x7f910c0c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f910a59c700 1 --2- 172.21.15.22:0/2624115293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f910c15ca60 0x7f910c0c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f910a59c700 1 -- 172.21.15.22:0/2624115293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9100009550 con 0x7f910c0ada90 2023-07-06T01:19:32.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f910a59c700 1 --2- 172.21.15.22:0/2624115293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f910c0ada90 0x7f910c0c3700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9100009870 tx=0x7f9100004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.050 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.980+0000 7f9f2757a700 1 Processor -- start 2023-07-06T01:19:32.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.981+0000 7f60a1745700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f609c160e50 0x7f609c0cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.981+0000 7f60a0f44700 1 -- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f609c160e50 msgr2=0x7f609c0cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.981+0000 7f60a0f44700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f609c160e50 0x7f609c0cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.981+0000 7f60a0f44700 1 -- 172.21.15.22:0/820607425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f609c0d0a90 msgr2=0x7f609c0cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.051 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.981+0000 7f60a0f44700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f609c0d0a90 0x7f609c0cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.981+0000 7f60a0f44700 1 -- 172.21.15.22:0/820607425 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60900094a0 con 0x7f609c0b0d10 2023-07-06T01:19:32.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.981+0000 7f9f2757a700 1 -- start start 2023-07-06T01:19:32.052 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.981+0000 7f60a0f44700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f609c0b0d10 0x7f609c15e4c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f608c00b470 tx=0x7f608c00b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f5dd2ffd700 1 -- 172.21.15.22:0/1778102195 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5dd4022440 con 0x7f5dcc004850 2023-07-06T01:19:32.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f60997fa700 1 -- 172.21.15.22:0/820607425 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c011830 con 0x7f609c0b0d10 2023-07-06T01:19:32.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7fee8e7fc700 1 -- 172.21.15.22:0/712024624 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88011830 con 0x7fee98161010 2023-07-06T01:19:32.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7fee8e7fc700 1 -- 172.21.15.22:0/712024624 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee880119b0 con 0x7fee98161010 2023-07-06T01:19:32.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f60997fa700 1 -- 172.21.15.22:0/820607425 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608c0119b0 con 0x7f609c0b0d10 2023-07-06T01:19:32.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f60997fa700 1 -- 172.21.15.22:0/820607425 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c011d60 con 0x7f609c0b0d10 2023-07-06T01:19:32.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f60a1745700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f609c160e50 0x7f609c0cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:32.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f8328ff9700 1 -- 172.21.15.22:0/170103422 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f832408e060 con 0x7f832c161010 2023-07-06T01:19:32.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7fee9ff90700 1 -- 172.21.15.22:0/712024624 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee980c4230 con 0x7fee98161010 2023-07-06T01:19:32.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7fee8e7fc700 1 -- 172.21.15.22:0/712024624 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88011d60 con 0x7fee98161010 2023-07-06T01:19:32.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f2757a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f0c004850 0x7f9f20155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f2757a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f20159a30 0x7f9f200c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f2757a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f200c8c60 0x7f9f200c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f2757a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2015f520 con 0x7f9f200c8c60 2023-07-06T01:19:32.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f2757a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20164170 con 0x7f9f0c004850 2023-07-06T01:19:32.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f2757a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2015f7e0 con 0x7f9f20159a30 2023-07-06T01:19:32.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f25316700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f0c004850 0x7f9f20155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f25316700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f0c004850 0x7f9f20155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:34352/0 (socket says 172.21.15.22:34352) 2023-07-06T01:19:32.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f25316700 1 -- 172.21.15.22:0/1559436987 learned_addr learned my addr 172.21.15.22:0/1559436987 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:32.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f24b15700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f20159a30 0x7f9f200c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f25b17700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f200c8c60 0x7f9f200c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f60a31a8700 1 -- 172.21.15.22:0/820607425 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609c15f160 con 0x7f609c0b0d10 2023-07-06T01:19:32.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f60a31a8700 1 -- 172.21.15.22:0/820607425 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609c15f720 con 0x7f609c0b0d10 2023-07-06T01:19:32.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f5dd2ffd700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5dc406ad30 0x7f5dc406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f5dd2ffd700 1 -- 172.21.15.22:0/1778102195 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5dd40899e0 con 0x7f5dcc004850 2023-07-06T01:19:32.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.983+0000 7f5de1d9b700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5dc406ad30 0x7f5dc406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.983+0000 7f5de1d9b700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5dc406ad30 0x7f5dc406d1e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8014040 tx=0x7f5dd8005040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.983+0000 7fee9ff90700 1 -- 172.21.15.22:0/712024624 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee980c47f0 con 0x7fee98161010 2023-07-06T01:19:32.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.984+0000 7f60a31a8700 1 -- 172.21.15.22:0/820607425 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609c0744b0 con 0x7f609c0b0d10 2023-07-06T01:19:32.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.984+0000 7fee9ff90700 1 -- 172.21.15.22:0/712024624 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee98068260 con 0x7fee98161010 2023-07-06T01:19:32.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f25316700 1 -- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f20159a30 msgr2=0x7f9f200c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f25316700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f20159a30 0x7f9f200c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f25316700 1 -- 172.21.15.22:0/1559436987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f200c8c60 msgr2=0x7f9f200c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f25316700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f200c8c60 0x7f9f200c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.982+0000 7f9f25316700 1 -- 172.21.15.22:0/1559436987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2015fac0 con 0x7f9f0c004850 2023-07-06T01:19:32.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.983+0000 7f9f25316700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f0c004850 0x7f9f20155e60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f1001daa0 tx=0x7f9f1001ddb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.983+0000 7f9f15ffb700 1 -- 172.21.15.22:0/1559436987 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10015c90 con 0x7f9f0c004850 2023-07-06T01:19:32.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.983+0000 7f9f15ffb700 1 -- 172.21.15.22:0/1559436987 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f10015df0 con 0x7f9f0c004850 2023-07-06T01:19:32.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.983+0000 7f9f2757a700 1 -- 172.21.15.22:0/1559436987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2015a5d0 con 0x7f9f0c004850 2023-07-06T01:19:32.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.983+0000 7f9f15ffb700 1 -- 172.21.15.22:0/1559436987 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10018310 con 0x7f9f0c004850 2023-07-06T01:19:32.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.983+0000 7f9f2757a700 1 -- 172.21.15.22:0/1559436987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f200c5e30 con 0x7f9f0c004850 2023-07-06T01:19:32.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.984+0000 7f9f2757a700 1 -- 172.21.15.22:0/1559436987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f20071600 con 0x7f9f0c004850 2023-07-06T01:19:32.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.985+0000 7f9f15ffb700 1 -- 172.21.15.22:0/1559436987 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9f10018470 con 0x7f9f0c004850 2023-07-06T01:19:32.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.985+0000 7f9f15ffb700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9f0406ace0 0x7f9f0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.985+0000 7f9f15ffb700 1 -- 172.21.15.22:0/1559436987 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9f10099630 con 0x7f9f0c004850 2023-07-06T01:19:32.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.986+0000 7f9f24b15700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9f0406ace0 0x7f9f0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.986+0000 7fd5476c6700 1 -- 172.21.15.22:0/1586803592 shutdown_connections 2023-07-06T01:19:32.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.986+0000 7fd5476c6700 1 -- 172.21.15.22:0/1586803592 wait complete. 2023-07-06T01:19:32.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.986+0000 7f9f24b15700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9f0406ace0 0x7f9f0406d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9f18017040 tx=0x7f9f18011000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.987+0000 7fd5476c6700 1 Processor -- start 2023-07-06T01:19:32.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.987+0000 7fd5476c6700 1 -- start start 2023-07-06T01:19:32.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7fd5476c6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd52c003b90 0x7fd54015d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7fd5476c6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5400c7f40 0x7fd5400c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7fd5476c6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540162a30 0x7fd5400cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7fd5476c6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400c5530 con 0x7fd5400c7f40 2023-07-06T01:19:32.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7fd5476c6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54015d7b0 con 0x7fd540162a30 2023-07-06T01:19:32.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7fd5476c6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400c6140 con 0x7fd52c003b90 2023-07-06T01:19:32.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7fd544c61700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5400c7f40 0x7fd5400c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7fd544c61700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5400c7f40 0x7fd5400c4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45640/0 (socket says 172.21.15.22:45640) 2023-07-06T01:19:32.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7fd544c61700 1 -- 172.21.15.22:0/3372680759 learned_addr learned my addr 172.21.15.22:0/3372680759 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:32.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7fd545c63700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540162a30 0x7fd5400cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7f90f37fe700 1 -- 172.21.15.22:0/2624115293 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91000036a0 con 0x7f910c0ada90 2023-07-06T01:19:32.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7f9111070700 1 -- 172.21.15.22:0/2624115293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910c0c4830 con 0x7f910c0ada90 2023-07-06T01:19:32.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7f9111070700 1 -- 172.21.15.22:0/2624115293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910c0b1d30 con 0x7f910c0ada90 2023-07-06T01:19:32.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.988+0000 7f5dd2ffd700 1 -- 172.21.15.22:0/1778102195 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5dd4054b70 con 0x7f5dcc004850 2023-07-06T01:19:32.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.989+0000 7f60997fa700 1 -- 172.21.15.22:0/820607425 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f608c01a400 con 0x7f609c0b0d10 2023-07-06T01:19:32.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.989+0000 7fd545462700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd52c003b90 0x7fd54015d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.067 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.989+0000 7fd544c61700 1 -- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd52c003b90 msgr2=0x7fd54015d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.989+0000 7fd544c61700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd52c003b90 0x7fd54015d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.989+0000 7fd544c61700 1 -- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540162a30 msgr2=0x7fd5400cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.989+0000 7fd544c61700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540162a30 0x7fd5400cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.989+0000 7fd544c61700 1 -- 172.21.15.22:0/3372680759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400c4650 con 0x7fd5400c7f40 2023-07-06T01:19:32.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.989+0000 7fd545462700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd52c003b90 0x7fd54015d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:32.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.989+0000 7f60997fa700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f608806ace0 0x7f608806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.989+0000 7f60997fa700 1 -- 172.21.15.22:0/820607425 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f608c089530 con 0x7f609c0b0d10 2023-07-06T01:19:32.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.989+0000 7f609bfff700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f608806ace0 0x7f608806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.990+0000 7f609bfff700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f608806ace0 0x7f608806d190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f609c06c6b0 tx=0x7f6090009f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.990+0000 7f9111070700 1 -- 172.21.15.22:0/2624115293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910c0744b0 con 0x7f910c0ada90 2023-07-06T01:19:32.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.990+0000 7fee8e7fc700 1 -- 172.21.15.22:0/712024624 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fee8801a400 con 0x7fee98161010 2023-07-06T01:19:32.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.990+0000 7fee8e7fc700 1 --2- 172.21.15.22:0/712024624 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee7c06ace0 0x7fee7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.990+0000 7fee9dd2c700 1 --2- 172.21.15.22:0/712024624 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee7c06ace0 0x7fee7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.990+0000 7f9f15ffb700 1 -- 172.21.15.22:0/1559436987 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9f20071600 con 0x7f9f0c004850 2023-07-06T01:19:32.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.991+0000 7fee9dd2c700 1 --2- 172.21.15.22:0/712024624 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee7c06ace0 0x7fee7c06d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fee94000c00 tx=0x7fee94021000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.992+0000 7f90f37fe700 1 -- 172.21.15.22:0/2624115293 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9100006d80 con 0x7f910c0ada90 2023-07-06T01:19:32.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.992+0000 7f90f37fe700 1 -- 172.21.15.22:0/2624115293 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100003a20 con 0x7f910c0ada90 2023-07-06T01:19:32.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.992+0000 7fd544c61700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5400c7f40 0x7fd5400c4110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd530000fb0 tx=0x7fd53000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.993+0000 7f90f37fe700 1 -- 172.21.15.22:0/2624115293 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9100022070 con 0x7f910c0ada90 2023-07-06T01:19:32.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.994+0000 7f909a7fc700 1 -- 172.21.15.22:0/46916284 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f909c010d60 con 0x7f9090006310 2023-07-06T01:19:32.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.994+0000 7f90f37fe700 1 --2- 172.21.15.22:0/2624115293 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f90f806ace0 0x7f90f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.994+0000 7f910ad9d700 1 --2- 172.21.15.22:0/2624115293 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f90f806ace0 0x7f90f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.994+0000 7f909a7fc700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f908806ace0 0x7f908806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.994+0000 7f909a7fc700 1 -- 172.21.15.22:0/46916284 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f909c08a770 con 0x7f9090006310 2023-07-06T01:19:32.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.994+0000 7f90a964f700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f908806ace0 0x7f908806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.995+0000 7f910ad9d700 1 --2- 172.21.15.22:0/2624115293 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f90f806ace0 0x7f90f806d190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9104009f60 tx=0x7f91040094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.995+0000 7f90a964f700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f908806ace0 0x7f908806d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f90a4156560 tx=0x7f9094008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.995+0000 7f90f37fe700 1 -- 172.21.15.22:0/2624115293 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f910005a160 con 0x7f910c0ada90 2023-07-06T01:19:32.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.995+0000 7fd535ffb700 1 -- 172.21.15.22:0/3372680759 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5300052e0 con 0x7fd5400c7f40 2023-07-06T01:19:32.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.996+0000 7fd5476c6700 1 -- 172.21.15.22:0/3372680759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5400c8440 con 0x7fd5400c7f40 2023-07-06T01:19:32.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.996+0000 7f909a7fc700 1 -- 172.21.15.22:0/46916284 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f909c055800 con 0x7f9090006310 2023-07-06T01:19:32.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.996+0000 7fd5476c6700 1 -- 172.21.15.22:0/3372680759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5400c63e0 con 0x7fd5400c7f40 2023-07-06T01:19:32.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.997+0000 7fee8e7fc700 1 -- 172.21.15.22:0/712024624 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fee8805b460 con 0x7fee98161010 2023-07-06T01:19:32.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:31.999+0000 7fee8e7fc700 1 -- 172.21.15.22:0/712024624 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fee880537e0 con 0x7fee98161010 2023-07-06T01:19:32.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.002+0000 7f60997fa700 1 -- 172.21.15.22:0/820607425 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f608c054560 con 0x7f609c0b0d10 2023-07-06T01:19:32.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.007+0000 7fd535ffb700 1 -- 172.21.15.22:0/3372680759 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd530005440 con 0x7fd5400c7f40 2023-07-06T01:19:32.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.007+0000 7fd535ffb700 1 -- 172.21.15.22:0/3372680759 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530003880 con 0x7fd5400c7f40 2023-07-06T01:19:32.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.008+0000 7fd535ffb700 1 -- 172.21.15.22:0/3372680759 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd530003a60 con 0x7fd5400c7f40 2023-07-06T01:19:32.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.008+0000 7fd535ffb700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd52406ad60 0x7fd52406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:32.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.008+0000 7fd535ffb700 1 -- 172.21.15.22:0/3372680759 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd530089350 con 0x7fd5400c7f40 2023-07-06T01:19:32.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.008+0000 7fd545462700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd52406ad60 0x7fd52406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:32.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.009+0000 7fd545462700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd52406ad60 0x7fd52406d210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd540162820 tx=0x7fd53c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:32.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.010+0000 7f90f37fe700 1 -- 172.21.15.22:0/2624115293 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f91000564a0 con 0x7f910c0ada90 2023-07-06T01:19:32.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.010+0000 7fd5476c6700 1 -- 172.21.15.22:0/3372680759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5200052f0 con 0x7fd5400c7f40 2023-07-06T01:19:32.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.019+0000 7fd535ffb700 1 -- 172.21.15.22:0/3372680759 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd530054400 con 0x7fd5400c7f40 2023-07-06T01:19:32.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.390+0000 7f5de88be700 1 -- 172.21.15.22:0/1778102195 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5dc8005870 con 0x7f5dcc004850 2023-07-06T01:19:32.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.390+0000 7f5dd2ffd700 1 -- 172.21.15.22:0/1778102195 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5dd401d090 con 0x7f5dcc004850 2023-07-06T01:19:32.395 INFO:teuthology.orchestra.run.smithi022.stdout:55834574852 2023-07-06T01:19:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.394+0000 7f5de88be700 1 -- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5dc406ad30 msgr2=0x7f5dc406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.394+0000 7f5de88be700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5dc406ad30 0x7f5dc406d1e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8014040 tx=0x7f5dd8005040 comp rx=0 tx=0).stop 2023-07-06T01:19:32.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.394+0000 7f5de88be700 1 -- 172.21.15.22:0/1778102195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dcc004850 msgr2=0x7f5de4154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.394+0000 7f5de88be700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dcc004850 0x7f5de4154950 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400fae0 tx=0x7f5dd400fdf0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.394+0000 7f5de88be700 1 -- 172.21.15.22:0/1778102195 shutdown_connections 2023-07-06T01:19:32.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.394+0000 7f5de88be700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5de40c66b0 0x7f5de41edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.394+0000 7f5de88be700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5dcc004850 0x7f5de4154950 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.394+0000 7f5de88be700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5dc406ad30 0x7f5dc406d1e0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8014040 tx=0x7f5dd8005040 comp rx=0 tx=0).stop 2023-07-06T01:19:32.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.394+0000 7f5de88be700 1 --2- 172.21.15.22:0/1778102195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5de40c6240 0x7f5de41ed780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.394+0000 7f5de88be700 1 -- 172.21.15.22:0/1778102195 >> 172.21.15.22:0/1778102195 conn(0x7f5de40b13a0 msgr2=0x7f5de40cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:32.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.394+0000 7f8332362700 1 -- 172.21.15.22:0/170103422 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8310005870 con 0x7f832c161010 2023-07-06T01:19:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.395+0000 7f5de88be700 1 -- 172.21.15.22:0/1778102195 shutdown_connections 2023-07-06T01:19:32.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.395+0000 7f8328ff9700 1 -- 172.21.15.22:0/170103422 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f83240542f0 con 0x7f832c161010 2023-07-06T01:19:32.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.395+0000 7f5de88be700 1 -- 172.21.15.22:0/1778102195 wait complete. 2023-07-06T01:19:32.401 INFO:teuthology.orchestra.run.smithi022.stdout:55834574852 2023-07-06T01:19:32.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.400+0000 7f8332362700 1 -- 172.21.15.22:0/170103422 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f830c06ace0 msgr2=0x7f830c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.400+0000 7f8332362700 1 --2- 172.21.15.22:0/170103422 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f830c06ace0 0x7f830c06d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8314000c00 tx=0x7f8314008000 comp rx=0 tx=0).stop 2023-07-06T01:19:32.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.400+0000 7f8332362700 1 -- 172.21.15.22:0/170103422 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f832c161010 msgr2=0x7f832c15ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.400+0000 7f8332362700 1 --2- 172.21.15.22:0/170103422 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f832c161010 0x7f832c15ef80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8324002000 tx=0x7f832400a340 comp rx=0 tx=0).stop 2023-07-06T01:19:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.401+0000 7f8332362700 1 -- 172.21.15.22:0/170103422 shutdown_connections 2023-07-06T01:19:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.401+0000 7f8332362700 1 --2- 172.21.15.22:0/170103422 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f832c161010 0x7f832c15ef80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.401+0000 7f8332362700 1 --2- 172.21.15.22:0/170103422 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f830c06ace0 0x7f830c06d190 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.401+0000 7f8332362700 1 --2- 172.21.15.22:0/170103422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f832c06d3a0 0x7f832c0b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.401+0000 7f8332362700 1 --2- 172.21.15.22:0/170103422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f832c06d810 0x7f832c0c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.401+0000 7f8332362700 1 -- 172.21.15.22:0/170103422 >> 172.21.15.22:0/170103422 conn(0x7f832c0b13a0 msgr2=0x7f832c0ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:32.408 INFO:teuthology.orchestra.run.smithi022.stdout:60129542149 2023-07-06T01:19:32.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.404+0000 7f9f2757a700 1 -- 172.21.15.22:0/1559436987 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9f20071600 con 0x7f9f0c004850 2023-07-06T01:19:32.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.404+0000 7f9f15ffb700 1 -- 172.21.15.22:0/1559436987 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9f20071600 con 0x7f9f0c004850 2023-07-06T01:19:32.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.407+0000 7f9f2757a700 1 -- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9f0406ace0 msgr2=0x7f9f0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.407+0000 7f9f2757a700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9f0406ace0 0x7f9f0406d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9f18017040 tx=0x7f9f18011000 comp rx=0 tx=0).stop 2023-07-06T01:19:32.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.407+0000 7f9f2757a700 1 -- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f0c004850 msgr2=0x7f9f20155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.407+0000 7f9f2757a700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f0c004850 0x7f9f20155e60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f1001daa0 tx=0x7f9f1001ddb0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.407+0000 7f9f2757a700 1 -- 172.21.15.22:0/1559436987 shutdown_connections 2023-07-06T01:19:32.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.407+0000 7f9f2757a700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9f200c8c60 0x7f9f200c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.407+0000 7f9f2757a700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9f0406ace0 0x7f9f0406d190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.407+0000 7f9f2757a700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f0c004850 0x7f9f20155e60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.407+0000 7f9f2757a700 1 --2- 172.21.15.22:0/1559436987 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f20159a30 0x7f9f200c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.407+0000 7f9f2757a700 1 -- 172.21.15.22:0/1559436987 >> 172.21.15.22:0/1559436987 conn(0x7f9f200b0710 msgr2=0x7f9f2006f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:32.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.408+0000 7f9f2757a700 1 -- 172.21.15.22:0/1559436987 shutdown_connections 2023-07-06T01:19:32.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.408+0000 7f9f2757a700 1 -- 172.21.15.22:0/1559436987 wait complete. 2023-07-06T01:19:32.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.409+0000 7f8332362700 1 -- 172.21.15.22:0/170103422 shutdown_connections 2023-07-06T01:19:32.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.413+0000 7f8332362700 1 -- 172.21.15.22:0/170103422 wait complete. 2023-07-06T01:19:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.413+0000 7f90ac0b4700 1 -- 172.21.15.22:0/46916284 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f90a41542d0 con 0x7f9090006310 2023-07-06T01:19:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.413+0000 7fee9ff90700 1 -- 172.21.15.22:0/712024624 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fee980c4cd0 con 0x7fee98161010 2023-07-06T01:19:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.413+0000 7fee8e7fc700 1 -- 172.21.15.22:0/712024624 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fee88015020 con 0x7fee98161010 2023-07-06T01:19:32.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.414+0000 7f909a7fc700 1 -- 172.21.15.22:0/46916284 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f909c058e20 con 0x7f9090006310 2023-07-06T01:19:32.416 INFO:teuthology.orchestra.run.smithi022.stdout:55834574852 2023-07-06T01:19:32.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2023-07-06T01:19:32.418 INFO:teuthology.orchestra.run.smithi022.stdout:60129542149 2023-07-06T01:19:32.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.417+0000 7fee9ff90700 1 -- 172.21.15.22:0/712024624 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee7c06ace0 msgr2=0x7fee7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.417+0000 7fee9ff90700 1 --2- 172.21.15.22:0/712024624 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee7c06ace0 0x7fee7c06d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fee94000c00 tx=0x7fee94021000 comp rx=0 tx=0).stop 2023-07-06T01:19:32.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.417+0000 7fee9ff90700 1 -- 172.21.15.22:0/712024624 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 msgr2=0x7fee980afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.417+0000 7fee9ff90700 1 --2- 172.21.15.22:0/712024624 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 0x7fee980afbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fee8800b440 tx=0x7fee8800b800 comp rx=0 tx=0).stop 2023-07-06T01:19:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.417+0000 7fee9ff90700 1 -- 172.21.15.22:0/712024624 shutdown_connections 2023-07-06T01:19:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.417+0000 7fee9ff90700 1 --2- 172.21.15.22:0/712024624 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee98161010 0x7fee980afbf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.417+0000 7fee9ff90700 1 --2- 172.21.15.22:0/712024624 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee7c06ace0 0x7fee7c06d190 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.417+0000 7fee9ff90700 1 --2- 172.21.15.22:0/712024624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee9806d3a0 0x7fee981571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.417+0000 7fee9ff90700 1 --2- 172.21.15.22:0/712024624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee9806d810 0x7fee980af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.417+0000 7fee9ff90700 1 -- 172.21.15.22:0/712024624 >> 172.21.15.22:0/712024624 conn(0x7fee980b13a0 msgr2=0x7fee98159e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.417+0000 7fee9ff90700 1 -- 172.21.15.22:0/712024624 shutdown_connections 2023-07-06T01:19:32.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.417+0000 7fee9ff90700 1 -- 172.21.15.22:0/712024624 wait complete. 2023-07-06T01:19:32.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.419+0000 7f60a31a8700 1 -- 172.21.15.22:0/820607425 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f609c0c9f90 con 0x7f609c0b0d10 2023-07-06T01:19:32.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.420+0000 7f90ac0b4700 1 -- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f908806ace0 msgr2=0x7f908806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.420+0000 7f90ac0b4700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f908806ace0 0x7f908806d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f90a4156560 tx=0x7f9094008040 comp rx=0 tx=0).stop 2023-07-06T01:19:32.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.420+0000 7f90ac0b4700 1 -- 172.21.15.22:0/46916284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9090006310 msgr2=0x7f90a4153d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.420+0000 7f90ac0b4700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9090006310 0x7f90a4153d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f909c007a90 tx=0x7f909c007da0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.420+0000 7f60997fa700 1 -- 172.21.15.22:0/820607425 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f608c057b80 con 0x7f609c0b0d10 2023-07-06T01:19:32.423 INFO:teuthology.orchestra.run.smithi022.stdout:55834574851 2023-07-06T01:19:32.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.422+0000 7f9111070700 1 -- 172.21.15.22:0/2624115293 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f910c0744b0 con 0x7f910c0ada90 2023-07-06T01:19:32.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.423+0000 7f90ac0b4700 1 -- 172.21.15.22:0/46916284 shutdown_connections 2023-07-06T01:19:32.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.423+0000 7f90ac0b4700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f908806ace0 0x7f908806d190 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.423+0000 7f90ac0b4700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90a4143400 0x7f90a40c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.423+0000 7f90ac0b4700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9090006310 0x7f90a4153d90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.423+0000 7f90ac0b4700 1 --2- 172.21.15.22:0/46916284 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90a40b18a0 0x7f90a4142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.423+0000 7f90ac0b4700 1 -- 172.21.15.22:0/46916284 >> 172.21.15.22:0/46916284 conn(0x7f90a406ac40 msgr2=0x7f90a40ae740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:32.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.423+0000 7f90f37fe700 1 -- 172.21.15.22:0/2624115293 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9100059ac0 con 0x7f910c0ada90 2023-07-06T01:19:32.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.425+0000 7f60a31a8700 1 -- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f608806ace0 msgr2=0x7f608806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.425+0000 7f60a31a8700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f608806ace0 0x7f608806d190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f609c06c6b0 tx=0x7f6090009f90 comp rx=0 tx=0).stop 2023-07-06T01:19:32.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.425+0000 7f60a31a8700 1 -- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f609c0b0d10 msgr2=0x7f609c15e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.425+0000 7f60a31a8700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f609c0b0d10 0x7f609c15e4c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f608c00b470 tx=0x7f608c00b830 comp rx=0 tx=0).stop 2023-07-06T01:19:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.425+0000 7f60a31a8700 1 -- 172.21.15.22:0/820607425 shutdown_connections 2023-07-06T01:19:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.425+0000 7f60a31a8700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f608806ace0 0x7f608806d190 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.425+0000 7f60a31a8700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f609c160e50 0x7f609c0cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.425+0000 7f60a31a8700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f609c0b0d10 0x7f609c15e4c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.425+0000 7f60a31a8700 1 --2- 172.21.15.22:0/820607425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f609c0d0a90 0x7f609c0cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.425+0000 7f60a31a8700 1 -- 172.21.15.22:0/820607425 >> 172.21.15.22:0/820607425 conn(0x7f609c09c2e0 msgr2=0x7f609c1e79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.426+0000 7f60a31a8700 1 -- 172.21.15.22:0/820607425 shutdown_connections 2023-07-06T01:19:32.429 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.4 2023-07-06T01:19:32.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.426+0000 7f60a31a8700 1 -- 172.21.15.22:0/820607425 wait complete. 2023-07-06T01:19:32.430 INFO:teuthology.orchestra.run.smithi022.stdout:55834574853 2023-07-06T01:19:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.429+0000 7f90ac0b4700 1 -- 172.21.15.22:0/46916284 shutdown_connections 2023-07-06T01:19:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.429+0000 7f90ac0b4700 1 -- 172.21.15.22:0/46916284 wait complete. 2023-07-06T01:19:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.429+0000 7f9111070700 1 -- 172.21.15.22:0/2624115293 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f90f806ace0 msgr2=0x7f90f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.429+0000 7f9111070700 1 --2- 172.21.15.22:0/2624115293 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f90f806ace0 0x7f90f806d190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9104009f60 tx=0x7f91040094f0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.429+0000 7f9111070700 1 -- 172.21.15.22:0/2624115293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f910c0ada90 msgr2=0x7f910c0c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.429+0000 7f9111070700 1 --2- 172.21.15.22:0/2624115293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f910c0ada90 0x7f910c0c3700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9100009870 tx=0x7f9100004750 comp rx=0 tx=0).stop 2023-07-06T01:19:32.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.429+0000 7f9111070700 1 -- 172.21.15.22:0/2624115293 shutdown_connections 2023-07-06T01:19:32.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.429+0000 7f9111070700 1 --2- 172.21.15.22:0/2624115293 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f90f806ace0 0x7f90f806d190 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9104009f60 tx=0x7f91040094f0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.429+0000 7f9111070700 1 --2- 172.21.15.22:0/2624115293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f910c15ca60 0x7f910c0c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.429+0000 7f9111070700 1 --2- 172.21.15.22:0/2624115293 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f910c06bda0 0x7f910c1ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.429+0000 7f9111070700 1 --2- 172.21.15.22:0/2624115293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f910c0ada90 0x7f910c0c3700 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.429+0000 7f9111070700 1 -- 172.21.15.22:0/2624115293 >> 172.21.15.22:0/2624115293 conn(0x7f910c0b0710 msgr2=0x7f910c0cb570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.431+0000 7f9111070700 1 -- 172.21.15.22:0/2624115293 shutdown_connections 2023-07-06T01:19:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.431+0000 7f9111070700 1 -- 172.21.15.22:0/2624115293 wait complete. 2023-07-06T01:19:32.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.437+0000 7fd5476c6700 1 -- 172.21.15.22:0/3372680759 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd5200055a0 con 0x7fd5400c7f40 2023-07-06T01:19:32.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.438+0000 7fd535ffb700 1 -- 172.21.15.22:0/3372680759 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd530057a20 con 0x7fd5400c7f40 2023-07-06T01:19:32.439 INFO:teuthology.orchestra.run.smithi022.stdout:55834574852 2023-07-06T01:19:32.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.441+0000 7fd5476c6700 1 -- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd52406ad60 msgr2=0x7fd52406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.441+0000 7fd5476c6700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd52406ad60 0x7fd52406d210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd540162820 tx=0x7fd53c008030 comp rx=0 tx=0).stop 2023-07-06T01:19:32.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.441+0000 7fd5476c6700 1 -- 172.21.15.22:0/3372680759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5400c7f40 msgr2=0x7fd5400c4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:32.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.441+0000 7fd5476c6700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5400c7f40 0x7fd5400c4110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd530000fb0 tx=0x7fd53000eb80 comp rx=0 tx=0).stop 2023-07-06T01:19:32.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2023-07-06T01:19:32.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2023-07-06T01:19:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.445+0000 7fd5476c6700 1 -- 172.21.15.22:0/3372680759 shutdown_connections 2023-07-06T01:19:32.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.445+0000 7fd5476c6700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd52406ad60 0x7fd52406d210 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.445+0000 7fd5476c6700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd540162a30 0x7fd5400cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.445+0000 7fd5476c6700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd52c003b90 0x7fd54015d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.445+0000 7fd5476c6700 1 --2- 172.21.15.22:0/3372680759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5400c7f40 0x7fd5400c4110 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.445+0000 7fd5476c6700 1 -- 172.21.15.22:0/3372680759 >> 172.21.15.22:0/3372680759 conn(0x7fd5400b0710 msgr2=0x7fd5400b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.445+0000 7fd5476c6700 1 -- 172.21.15.22:0/3372680759 shutdown_connections 2023-07-06T01:19:32.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:32.445+0000 7fd5476c6700 1 -- 172.21.15.22:0/3372680759 wait complete. 2023-07-06T01:19:32.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.7 2023-07-06T01:19:32.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2023-07-06T01:19:32.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-07-06T01:19:32.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2023-07-06T01:19:33.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-06T01:19:33.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-06T01:19:33.445 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-06T01:19:33.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-06T01:19:33.449 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-06T01:19:33.450 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-06T01:19:33.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-06T01:19:33.461 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-06T01:19:33.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.611+0000 7f6d6bb13700 1 Processor -- start 2023-07-06T01:19:33.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.611+0000 7f6d6bb13700 1 -- start start 2023-07-06T01:19:33.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.613+0000 7f6d6bb13700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f6d6406d3a0 0x7f6d6406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.613+0000 7f6d6bb13700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d6408a220 0x7f6d6416d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.613+0000 7f6d6bb13700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d640cb430 con 0x7f6d64169940 2023-07-06T01:19:33.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.613+0000 7f6d6bb13700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d640cab00 con 0x7f6d6406d3a0 2023-07-06T01:19:33.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.613+0000 7f6d6bb13700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640cade0 con 0x7f6d6408a220 2023-07-06T01:19:33.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.614+0000 7f6d6a0b0700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d6408a220 0x7f6d6416d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.614+0000 7f6d6a0b0700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d6408a220 0x7f6d6416d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56794/0 (socket says 172.21.15.22:56794) 2023-07-06T01:19:33.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.614+0000 7f6d6a0b0700 1 -- 172.21.15.22:0/2887141356 learned_addr learned my addr 172.21.15.22:0/2887141356 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:33.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.614+0000 7f6d698af700 1 --2- 172.21.15.22:0/2887141356 >> v2:172.21.15.103:3300/0 conn(0x7f6d6406d3a0 0x7f6d6406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.615+0000 7f6d688ad700 1 -- 172.21.15.22:0/2887141356 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635593610 0 0) 0x7f6d640cb430 con 0x7f6d64169940 2023-07-06T01:19:33.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.615+0000 7f6d688ad700 1 -- 172.21.15.22:0/2887141356 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d500079c0 con 0x7f6d64169940 2023-07-06T01:19:33.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.616+0000 7f6d6a0b0700 1 -- 172.21.15.22:0/2887141356 >> v2:172.21.15.103:3300/0 conn(0x7f6d6406d3a0 msgr2=0x7f6d6406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:33.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.616+0000 7f6d6a0b0700 1 --2- 172.21.15.22:0/2887141356 >> v2:172.21.15.103:3300/0 conn(0x7f6d6406d3a0 0x7f6d6406b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.616+0000 7f6d6a0b0700 1 -- 172.21.15.22:0/2887141356 >> v1:172.21.15.103:6789/0 conn(0x7f6d64169940 legacy=0x7f6d6406d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.616+0000 7f6d6a0b0700 1 -- 172.21.15.22:0/2887141356 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d640cb0c0 con 0x7f6d6408a220 2023-07-06T01:19:33.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.616+0000 7f6d6a0b0700 1 --2- 172.21.15.22:0/2887141356 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d6408a220 0x7f6d6416d4d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6d60008370 tx=0x7f6d6000ac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5dda0c7fb5aa025 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.623 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.616+0000 7f6d688ad700 1 -- 172.21.15.22:0/2887141356 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60008df0 con 0x7f6d6408a220 2023-07-06T01:19:33.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.616+0000 7f6d688ad700 1 -- 172.21.15.22:0/2887141356 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6000a0d0 con 0x7f6d6408a220 2023-07-06T01:19:33.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.616+0000 7f6d688ad700 1 -- 172.21.15.22:0/2887141356 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6000a440 con 0x7f6d6408a220 2023-07-06T01:19:33.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.617+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2887141356 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d6408a220 msgr2=0x7f6d6416d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.617+0000 7f6d6bb13700 1 --2- 172.21.15.22:0/2887141356 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d6408a220 0x7f6d6416d4d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6d60008370 tx=0x7f6d6000ac60 comp rx=0 tx=0).stop 2023-07-06T01:19:33.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.617+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2887141356 shutdown_connections 2023-07-06T01:19:33.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.617+0000 7f6d6bb13700 1 --2- 172.21.15.22:0/2887141356 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d6408a220 0x7f6d6416d4d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.617+0000 7f6d6bb13700 1 --2- 172.21.15.22:0/2887141356 >> v2:172.21.15.103:3300/0 conn(0x7f6d6406d3a0 0x7f6d6406b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.617+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2887141356 >> 172.21.15.22:0/2887141356 conn(0x7f6d640b13a0 msgr2=0x7f6d640ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:33.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.618+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2887141356 shutdown_connections 2023-07-06T01:19:33.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.618+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2887141356 wait complete. 2023-07-06T01:19:33.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.620+0000 7f6d6bb13700 1 Processor -- start 2023-07-06T01:19:33.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.620+0000 7f6d6bb13700 1 -- start start 2023-07-06T01:19:33.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.621+0000 7f6d6bb13700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d6406d3a0 0x7f6d64165010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.621+0000 7f6d6bb13700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d6408a220 0x7f6d640c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.621+0000 7f6d6bb13700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d64169940 0x7f6d640c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.621+0000 7f6d6bb13700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640c6080 con 0x7f6d6406d3a0 2023-07-06T01:19:33.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.621+0000 7f6d6bb13700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640c46b0 con 0x7f6d6408a220 2023-07-06T01:19:33.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.621+0000 7f6d6bb13700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640c4990 con 0x7f6d64169940 2023-07-06T01:19:33.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.622+0000 7f6d690ae700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d6408a220 0x7f6d640c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.622+0000 7f6d6a0b0700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d64169940 0x7f6d640c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.623+0000 7f6d698af700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d6406d3a0 0x7f6d64165010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.623+0000 7f6d698af700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d6406d3a0 0x7f6d64165010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45662/0 (socket says 172.21.15.22:45662) 2023-07-06T01:19:33.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.623+0000 7f6d698af700 1 -- 172.21.15.22:0/2553154693 learned_addr learned my addr 172.21.15.22:0/2553154693 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:33.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.623+0000 7f6d698af700 1 -- 172.21.15.22:0/2553154693 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d64169940 msgr2=0x7f6d640c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.623+0000 7f6d698af700 1 --2- 172.21.15.22:0/2553154693 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d64169940 0x7f6d640c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.623+0000 7f6d698af700 1 -- 172.21.15.22:0/2553154693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d6408a220 msgr2=0x7f6d640c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.623+0000 7f6d698af700 1 --2- 172.21.15.22:0/2553154693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d6408a220 0x7f6d640c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.623+0000 7f6d698af700 1 -- 172.21.15.22:0/2553154693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d60008050 con 0x7f6d6406d3a0 2023-07-06T01:19:33.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.623+0000 7f6d698af700 1 --2- 172.21.15.22:0/2553154693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d6406d3a0 0x7f6d64165010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c007e40 tx=0x7f6d5c0085d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.626+0000 7f6d5a7fc700 1 -- 172.21.15.22:0/2553154693 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c011040 con 0x7f6d6406d3a0 2023-07-06T01:19:33.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.626+0000 7f6d5a7fc700 1 -- 172.21.15.22:0/2553154693 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c00c2d0 con 0x7f6d6406d3a0 2023-07-06T01:19:33.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.626+0000 7f6d5a7fc700 1 -- 172.21.15.22:0/2553154693 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00a6e0 con 0x7f6d6406d3a0 2023-07-06T01:19:33.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.626+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2553154693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d640c4c70 con 0x7f6d6406d3a0 2023-07-06T01:19:33.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.626+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2553154693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d64165550 con 0x7f6d6406d3a0 2023-07-06T01:19:33.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.633+0000 7f6d5a7fc700 1 -- 172.21.15.22:0/2553154693 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6d5c008040 con 0x7f6d6406d3a0 2023-07-06T01:19:33.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.633+0000 7f6d5a7fc700 1 --2- 172.21.15.22:0/2553154693 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6d4806ade0 0x7f6d4806d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.634+0000 7f6d5a7fc700 1 -- 172.21.15.22:0/2553154693 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6d5c0897e0 con 0x7f6d6406d3a0 2023-07-06T01:19:33.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.636+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2553154693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4c0052f0 con 0x7f6d6406d3a0 2023-07-06T01:19:33.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.641+0000 7f6d690ae700 1 --2- 172.21.15.22:0/2553154693 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6d4806ade0 0x7f6d4806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.646+0000 7f6d5a7fc700 1 -- 172.21.15.22:0/2553154693 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6d5c054900 con 0x7f6d6406d3a0 2023-07-06T01:19:33.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.648+0000 7f6d690ae700 1 --2- 172.21.15.22:0/2553154693 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6d4806ade0 0x7f6d4806d290 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d540010e0 tx=0x7f6d54009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.695+0000 7f3b5ae4e700 1 Processor -- start 2023-07-06T01:19:33.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.693+0000 7f7215035700 1 Processor -- start 2023-07-06T01:19:33.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.693+0000 7f7215035700 1 -- start start 2023-07-06T01:19:33.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.694+0000 7f7215035700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f721006d3a0 0x7f721006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.694+0000 7f7215035700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f721008a220 0x7f721016d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.694+0000 7f7215035700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72100cb430 con 0x7f7210169940 2023-07-06T01:19:33.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.694+0000 7f7215035700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72100cab00 con 0x7f721006d3a0 2023-07-06T01:19:33.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.694+0000 7f7215035700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72100cade0 con 0x7f721008a220 2023-07-06T01:19:33.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.695+0000 7f720ed9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f721006d3a0 0x7f721006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.695+0000 7f720ed9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f721006d3a0 0x7f721006b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45670/0 (socket says 172.21.15.22:45670) 2023-07-06T01:19:33.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.695+0000 7f720ed9d700 1 -- 172.21.15.22:0/1276257923 learned_addr learned my addr 172.21.15.22:0/1276257923 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:33.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.696+0000 7f720dd9b700 1 -- 172.21.15.22:0/1276257923 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059852261 0 0) 0x7f72100cb430 con 0x7f7210169940 2023-07-06T01:19:33.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.695+0000 7f3b5ae4e700 1 -- start start 2023-07-06T01:19:33.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.697+0000 7f3b5ae4e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3b540cac80 0x7f3b5406c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.697+0000 7f3b5ae4e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5415d4e0 con 0x7f3b540c91d0 2023-07-06T01:19:33.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.697+0000 7f3b5ae4e700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b540cdd50 con 0x7f3b5406d3a0 2023-07-06T01:19:33.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.697+0000 7f3b5ae4e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b540cb410 con 0x7f3b540cac80 2023-07-06T01:19:33.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.697+0000 7f3b58bea700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f3b5406d3a0 0x7f3b5406d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38482/0 (socket says 172.21.15.22:38482) 2023-07-06T01:19:33.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.697+0000 7f3b593eb700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3b540cac80 0x7f3b5406c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.697+0000 7f3b58bea700 1 -- 172.21.15.22:0/3556877044 learned_addr learned my addr 172.21.15.22:0/3556877044 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:33.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.697+0000 7f720dd9b700 1 -- 172.21.15.22:0/1276257923 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71f8003a00 con 0x7f7210169940 2023-07-06T01:19:33.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.697+0000 7f3b593eb700 1 -- 172.21.15.22:0/3556877044 >> v1:172.21.15.103:6789/0 conn(0x7f3b5406d3a0 legacy=0x7f3b5406d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.697+0000 7f3b593eb700 1 -- 172.21.15.22:0/3556877044 >> v1:172.21.15.22:6789/0 conn(0x7f3b540c91d0 legacy=0x7f3b5406b9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T01:19:33.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.697+0000 7f3b593eb700 1 -- 172.21.15.22:0/3556877044 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5415c720 con 0x7f3b540cac80 2023-07-06T01:19:33.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.698+0000 7f720ed9d700 1 -- 172.21.15.22:0/1276257923 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f721008a220 msgr2=0x7f721016d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:33.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.698+0000 7f3b593eb700 1 --2- 172.21.15.22:0/3556877044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cac80 0x7f3b5406c0f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c006600 tx=0x7f3b4c007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39090c789e0043c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.698+0000 7f720dd9b700 1 -- 172.21.15.22:0/1276257923 <== mon.1 v1:172.21.15.103:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2327964760 0 0) 0x7f71f8003a00 con 0x7f7210169940 2023-07-06T01:19:33.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.698+0000 7f3b537fe700 1 -- 172.21.15.22:0/3556877044 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c003c20 con 0x7f3b540cac80 2023-07-06T01:19:33.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.698+0000 7f3b537fe700 1 -- 172.21.15.22:0/3556877044 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c003d80 con 0x7f3b540cac80 2023-07-06T01:19:33.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.698+0000 7f3b537fe700 1 -- 172.21.15.22:0/3556877044 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c013bc0 con 0x7f3b540cac80 2023-07-06T01:19:33.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.699+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/3556877044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cac80 msgr2=0x7f3b5406c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.699+0000 7f3b5ae4e700 1 --2- 172.21.15.22:0/3556877044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cac80 0x7f3b5406c0f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c006600 tx=0x7f3b4c007e60 comp rx=0 tx=0).stop 2023-07-06T01:19:33.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.699+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/3556877044 shutdown_connections 2023-07-06T01:19:33.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.699+0000 7f3b5ae4e700 1 --2- 172.21.15.22:0/3556877044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540cac80 0x7f3b5406c0f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.699+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/3556877044 >> 172.21.15.22:0/3556877044 conn(0x7f3b540b13a0 msgr2=0x7f3b540ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:33.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.695+0000 7f720f59e700 1 --2- 172.21.15.22:0/1276257923 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f721008a220 0x7f721016d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.698+0000 7faf8e2b6700 1 Processor -- start 2023-07-06T01:19:33.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.698+0000 7faf8e2b6700 1 -- start start 2023-07-06T01:19:33.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7f720ed9d700 1 --2- 172.21.15.22:0/1276257923 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f721008a220 0x7f721016d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7f720ed9d700 1 -- 172.21.15.22:0/1276257923 >> v1:172.21.15.103:6789/0 conn(0x7f7210169940 legacy=0x7f721006d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7f720ed9d700 1 -- 172.21.15.22:0/1276257923 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72100cb0c0 con 0x7f721006d3a0 2023-07-06T01:19:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7f720f59e700 1 --2- 172.21.15.22:0/1276257923 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f721008a220 0x7f721016d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7f720ed9d700 1 --2- 172.21.15.22:0/1276257923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f721006d3a0 0x7f721006b9e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f72040092e0 tx=0x7f72040095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e35caf3826807ff server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7faf8e2b6700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7faf8808a220 0x7faf881696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7faf8e2b6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faf8808a770 0x7faf8806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7faf8e2b6700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf880cb430 con 0x7faf8806d3a0 2023-07-06T01:19:33.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7faf8e2b6700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf880cade0 con 0x7faf8808a220 2023-07-06T01:19:33.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7faf8e2b6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf880cb0c0 con 0x7faf8808a770 2023-07-06T01:19:33.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7f720dd9b700 1 -- 172.21.15.22:0/1276257923 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200001670 con 0x7f721006d3a0 2023-07-06T01:19:33.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7faf87fff700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7faf8808a220 0x7faf881696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7faf8c853700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7faf8806d3a0 0x7faf8816d4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38490/0 (socket says 172.21.15.22:38490) 2023-07-06T01:19:33.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7faf8c853700 1 -- 172.21.15.22:0/3977437586 learned_addr learned my addr 172.21.15.22:0/3977437586 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:33.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7f720dd9b700 1 -- 172.21.15.22:0/1276257923 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7204004030 con 0x7f721006d3a0 2023-07-06T01:19:33.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7f720dd9b700 1 -- 172.21.15.22:0/1276257923 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72040043c0 con 0x7f721006d3a0 2023-07-06T01:19:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191790703 0 0) 0x7faf880cb430 con 0x7faf8806d3a0 2023-07-06T01:19:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.700+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf700035b0 con 0x7faf8806d3a0 2023-07-06T01:19:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf877fe700 1 --2- 172.21.15.22:0/3977437586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faf8808a770 0x7faf8806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 <== mon.1 v1:172.21.15.103:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2310622498 0 0) 0x7faf700035b0 con 0x7faf8806d3a0 2023-07-06T01:19:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faf8808a770 msgr2=0x7faf8806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 --2- 172.21.15.22:0/3977437586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faf8808a770 0x7faf8806b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 >> v2:172.21.15.103:3300/0 conn(0x7faf8808a220 msgr2=0x7faf881696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:33.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 --2- 172.21.15.22:0/3977437586 >> v2:172.21.15.103:3300/0 conn(0x7faf8808a220 0x7faf881696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 --> v1:172.21.15.103:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf880ca7b0 con 0x7faf8806d3a0 2023-07-06T01:19:33.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 <== mon.1 v1:172.21.15.103:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7faf800019f0 con 0x7faf8806d3a0 2023-07-06T01:19:33.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 >> v1:172.21.15.103:6789/0 conn(0x7faf8806d3a0 legacy=0x7faf8816d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 --2- 172.21.15.22:0/3977437586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf70003a00 0x7faf70003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 --2- 172.21.15.22:0/3977437586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faf8808a770 0x7faf70004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 --2- 172.21.15.22:0/3977437586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf70004d90 0x7faf70005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf880cade0 con 0x7faf70004d90 2023-07-06T01:19:33.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf880cb0c0 con 0x7faf70003a00 2023-07-06T01:19:33.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7000ba10 con 0x7faf8808a770 2023-07-06T01:19:33.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf87fff700 1 --2- 172.21.15.22:0/3977437586 >> v2:172.21.15.103:3300/0 conn(0x7faf8808a220 0x7faf881696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:33.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf8c853700 1 --2- 172.21.15.22:0/3977437586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf70004d90 0x7faf70005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf877fe700 1 --2- 172.21.15.22:0/3977437586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faf8808a770 0x7faf70004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7faf87fff700 1 --2- 172.21.15.22:0/3977437586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf70003a00 0x7faf70003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7f7215035700 1 -- 172.21.15.22:0/1276257923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f721006d3a0 msgr2=0x7f721006b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7f7215035700 1 --2- 172.21.15.22:0/1276257923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f721006d3a0 0x7f721006b9e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f72040092e0 tx=0x7f72040095f0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7f7215035700 1 -- 172.21.15.22:0/1276257923 shutdown_connections 2023-07-06T01:19:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7f7215035700 1 --2- 172.21.15.22:0/1276257923 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f721008a220 0x7f721016d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7f7215035700 1 --2- 172.21.15.22:0/1276257923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f721006d3a0 0x7f721006b9e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7f7215035700 1 -- 172.21.15.22:0/1276257923 >> 172.21.15.22:0/1276257923 conn(0x7f72100b13a0 msgr2=0x7f72100ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7f7215035700 1 -- 172.21.15.22:0/1276257923 shutdown_connections 2023-07-06T01:19:33.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.701+0000 7f7215035700 1 -- 172.21.15.22:0/1276257923 wait complete. 2023-07-06T01:19:33.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.702+0000 7faf877fe700 1 -- 172.21.15.22:0/3977437586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf70003a00 msgr2=0x7faf70003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:33.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.702+0000 7faf877fe700 1 --2- 172.21.15.22:0/3977437586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf70003a00 0x7faf70003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.702+0000 7faf877fe700 1 -- 172.21.15.22:0/3977437586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf70004d90 msgr2=0x7faf70005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:33.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.702+0000 7faf877fe700 1 --2- 172.21.15.22:0/3977437586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf70004d90 0x7faf70005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.702+0000 7faf877fe700 1 -- 172.21.15.22:0/3977437586 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf880ca7b0 con 0x7faf8808a770 2023-07-06T01:19:33.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.702+0000 7faf8c853700 1 --2- 172.21.15.22:0/3977437586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf70004d90 0x7faf70005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:33.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.702+0000 7faf877fe700 1 --2- 172.21.15.22:0/3977437586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faf8808a770 0x7faf70004850 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009890 tx=0x7faf70005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2cadb895cfd44c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.702+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00c070 con 0x7faf8808a770 2023-07-06T01:19:33.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.702+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c013610 con 0x7faf8808a770 2023-07-06T01:19:33.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.702+0000 7faf86ffd700 1 -- 172.21.15.22:0/3977437586 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c013980 con 0x7faf8808a770 2023-07-06T01:19:33.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.703+0000 7faf8e2b6700 1 -- 172.21.15.22:0/3977437586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faf8808a770 msgr2=0x7faf70004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.703+0000 7faf8e2b6700 1 --2- 172.21.15.22:0/3977437586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faf8808a770 0x7faf70004850 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009890 tx=0x7faf70005880 comp rx=0 tx=0).stop 2023-07-06T01:19:33.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.703+0000 7faf8c853700 1 -- 172.21.15.22:0/3977437586 reap_dead start 2023-07-06T01:19:33.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.703+0000 7faf8e2b6700 1 -- 172.21.15.22:0/3977437586 shutdown_connections 2023-07-06T01:19:33.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.703+0000 7faf8e2b6700 1 -- 172.21.15.22:0/3977437586 >> 172.21.15.22:0/3977437586 conn(0x7faf880b13a0 msgr2=0x7faf880ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.704+0000 7f6e3d15b700 1 Processor -- start 2023-07-06T01:19:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.704+0000 7faf8e2b6700 1 -- 172.21.15.22:0/3977437586 shutdown_connections 2023-07-06T01:19:33.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.704+0000 7f6e3d15b700 1 -- start start 2023-07-06T01:19:33.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.704+0000 7f7215035700 1 Processor -- start 2023-07-06T01:19:33.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.704+0000 7f7215035700 1 -- start start 2023-07-06T01:19:33.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f7215035700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f721006d3a0 0x7f7210086fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f7215035700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f721008a220 0x7f7210085140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f7215035700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7210169940 0x7f7210085680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f7215035700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72100878e0 con 0x7f721006d3a0 2023-07-06T01:19:33.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f7215035700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7210087bc0 con 0x7f721008a220 2023-07-06T01:19:33.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f7215035700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7210087ea0 con 0x7f7210169940 2023-07-06T01:19:33.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f6e3d15b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6e381597a0 0x7f6e3806cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f6e3d15b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6e380cc750 0x7f6e380cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f6e3d15b700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e380ccf30 con 0x7f6e380c9a50 2023-07-06T01:19:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f6e3d15b700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e380cd210 con 0x7f6e381597a0 2023-07-06T01:19:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f6e3d15b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e380cd4f0 con 0x7f6e380cc750 2023-07-06T01:19:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7f720ed9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f721006d3a0 0x7f7210086fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7f720ed9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f721006d3a0 0x7f7210086fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45694/0 (socket says 172.21.15.22:45694) 2023-07-06T01:19:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7f720ed9d700 1 -- 172.21.15.22:0/1194700318 learned_addr learned my addr 172.21.15.22:0/1194700318 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:33.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f6e3659c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6e380cc750 0x7f6e380cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f6e3759e700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f6e380c9a50 0x7f6e3806fca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38500/0 (socket says 172.21.15.22:38500) 2023-07-06T01:19:33.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7f6e3759e700 1 -- 172.21.15.22:0/211695554 learned_addr learned my addr 172.21.15.22:0/211695554 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:33.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7f6e3659c700 1 -- 172.21.15.22:0/211695554 >> v2:172.21.15.22:3300/0 conn(0x7f6e381597a0 msgr2=0x7f6e3806cf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T01:19:33.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7f6e3659c700 1 --2- 172.21.15.22:0/211695554 >> v2:172.21.15.22:3300/0 conn(0x7f6e381597a0 0x7f6e3806cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7f6e3659c700 1 -- 172.21.15.22:0/211695554 >> v1:172.21.15.103:6789/0 conn(0x7f6e380c9a50 legacy=0x7f6e3806fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7f6e3659c700 1 -- 172.21.15.22:0/211695554 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e380cd7d0 con 0x7f6e380cc750 2023-07-06T01:19:33.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/3556877044 shutdown_connections 2023-07-06T01:19:33.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7f6e3659c700 1 --2- 172.21.15.22:0/211695554 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6e380cc750 0x7f6e380cdfd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000be30 tx=0x7f6e30010730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12a974fd1bd99ced server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7faf8e2b6700 1 -- 172.21.15.22:0/3977437586 wait complete. 2023-07-06T01:19:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7fbbb5c48700 1 Processor -- start 2023-07-06T01:19:33.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.705+0000 7fbbb5c48700 1 -- start start 2023-07-06T01:19:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7fbbb5c48700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbbb006d3a0 0x7fbbb006d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7fbbb5c48700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fbbb006b9e0 0x7fbbb006be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7fbbb5c48700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb00714d0 con 0x7fbbb0161010 2023-07-06T01:19:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7fbbb5c48700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb006dd50 con 0x7fbbb006d3a0 2023-07-06T01:19:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7fbbb5c48700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb00717b0 con 0x7fbbb006b9e0 2023-07-06T01:19:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.707+0000 7fbbaffff700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fbbb006b9e0 0x7fbbb006be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.707+0000 7fbbaffff700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fbbb006b9e0 0x7fbbb006be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:34426/0 (socket says 172.21.15.22:34426) 2023-07-06T01:19:33.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.707+0000 7fbbaffff700 1 -- 172.21.15.22:0/2037761757 learned_addr learned my addr 172.21.15.22:0/2037761757 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:33.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.707+0000 7faf8e2b6700 1 Processor -- start 2023-07-06T01:19:33.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7f6e35d9b700 1 -- 172.21.15.22:0/211695554 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30003a20 con 0x7f6e380cc750 2023-07-06T01:19:33.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7f6e35d9b700 1 -- 172.21.15.22:0/211695554 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e30003b80 con 0x7f6e380cc750 2023-07-06T01:19:33.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7f6e35d9b700 1 -- 172.21.15.22:0/211695554 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30004030 con 0x7f6e380cc750 2023-07-06T01:19:33.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.707+0000 7f6e3d15b700 1 -- 172.21.15.22:0/211695554 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6e380cc750 msgr2=0x7f6e380cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.707+0000 7f6e3d15b700 1 --2- 172.21.15.22:0/211695554 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6e380cc750 0x7f6e380cdfd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000be30 tx=0x7f6e30010730 comp rx=0 tx=0).stop 2023-07-06T01:19:33.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.707+0000 7fbbaffff700 1 -- 172.21.15.22:0/2037761757 >> v2:172.21.15.22:3300/0 conn(0x7fbbb006d3a0 msgr2=0x7fbbb006d810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T01:19:33.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.707+0000 7fbbaffff700 1 --2- 172.21.15.22:0/2037761757 >> v2:172.21.15.22:3300/0 conn(0x7fbbb006d3a0 0x7fbbb006d810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.707+0000 7fbbaffff700 1 -- 172.21.15.22:0/2037761757 >> v1:172.21.15.103:6789/0 conn(0x7fbbb0161010 legacy=0x7fbbb00cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.707+0000 7fbbaffff700 1 -- 172.21.15.22:0/2037761757 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb0071a90 con 0x7fbbb006b9e0 2023-07-06T01:19:33.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.706+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/3556877044 wait complete. 2023-07-06T01:19:33.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f6e3d15b700 1 -- 172.21.15.22:0/211695554 shutdown_connections 2023-07-06T01:19:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f6e3d15b700 1 --2- 172.21.15.22:0/211695554 >> v2:172.21.15.22:3300/0 conn(0x7f6e381597a0 0x7f6e3806cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f6e3d15b700 1 --2- 172.21.15.22:0/211695554 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6e380cc750 0x7f6e380cdfd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f6e3d15b700 1 -- 172.21.15.22:0/211695554 >> 172.21.15.22:0/211695554 conn(0x7f6e380b0710 msgr2=0x7f6e380b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f6e3d15b700 1 -- 172.21.15.22:0/211695554 shutdown_connections 2023-07-06T01:19:33.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7fbbaffff700 1 --2- 172.21.15.22:0/2037761757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb006b9e0 0x7fbbb006be20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbbb006b4a0 tx=0x7fbba0009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9f5cff5aa7c1d1d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f3b5ae4e700 1 Processor -- start 2023-07-06T01:19:33.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f720ed9d700 1 -- 172.21.15.22:0/1194700318 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7210169940 msgr2=0x7f7210085680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T01:19:33.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f720ed9d700 1 --2- 172.21.15.22:0/1194700318 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7210169940 0x7f7210085680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f720ed9d700 1 -- 172.21.15.22:0/1194700318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f721008a220 msgr2=0x7f7210085140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T01:19:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f720ed9d700 1 --2- 172.21.15.22:0/1194700318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f721008a220 0x7f7210085140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f720ed9d700 1 -- 172.21.15.22:0/1194700318 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7204009de0 con 0x7f721006d3a0 2023-07-06T01:19:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f720ed9d700 1 --2- 172.21.15.22:0/1194700318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f721006d3a0 0x7f7210086fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7210074000 tx=0x7f7204004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.707+0000 7faf8e2b6700 1 -- start start 2023-07-06T01:19:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7faf8e2b6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf88165730 0x7faf881795d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7faf8e2b6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faf88165b00 0x7faf880c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7faf8e2b6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf880c33f0 0x7faf880c1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7faf8e2b6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf880b3530 con 0x7faf88165730 2023-07-06T01:19:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7faf8e2b6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf880c39b0 con 0x7faf880c33f0 2023-07-06T01:19:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7faf8e2b6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf880c3c90 con 0x7faf88165b00 2023-07-06T01:19:33.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7fbbae7fc700 1 -- 172.21.15.22:0/2037761757 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba000b870 con 0x7fbbb006b9e0 2023-07-06T01:19:33.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7fbbae7fc700 1 -- 172.21.15.22:0/2037761757 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba000b9d0 con 0x7fbbb006b9e0 2023-07-06T01:19:33.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7fbbae7fc700 1 -- 172.21.15.22:0/2037761757 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba000bd40 con 0x7fbbb006b9e0 2023-07-06T01:19:33.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7fbbb5c48700 1 -- 172.21.15.22:0/2037761757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb006b9e0 msgr2=0x7fbbb006be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7fbbb5c48700 1 --2- 172.21.15.22:0/2037761757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb006b9e0 0x7fbbb006be20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbbb006b4a0 tx=0x7fbba0009bd0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7fbbb5c48700 1 -- 172.21.15.22:0/2037761757 shutdown_connections 2023-07-06T01:19:33.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7fbbb5c48700 1 --2- 172.21.15.22:0/2037761757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb006b9e0 0x7fbbb006be20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7fbbb5c48700 1 --2- 172.21.15.22:0/2037761757 >> v2:172.21.15.22:3300/0 conn(0x7fbbb006d3a0 0x7fbbb006d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7fbbb5c48700 1 -- 172.21.15.22:0/2037761757 >> 172.21.15.22:0/2037761757 conn(0x7fbbb00b13a0 msgr2=0x7fbbb00ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:33.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7f71f77fe700 1 -- 172.21.15.22:0/1194700318 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720401e070 con 0x7f721006d3a0 2023-07-06T01:19:33.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7f71f77fe700 1 -- 172.21.15.22:0/1194700318 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720400c050 con 0x7f721006d3a0 2023-07-06T01:19:33.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7f71f77fe700 1 -- 172.21.15.22:0/1194700318 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204010a10 con 0x7f721006d3a0 2023-07-06T01:19:33.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7f7215035700 1 -- 172.21.15.22:0/1194700318 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7210088180 con 0x7f721006d3a0 2023-07-06T01:19:33.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f3b5ae4e700 1 -- start start 2023-07-06T01:19:33.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b5ae4e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3b5406d3a0 0x7f3b54158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b5ae4e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540c91d0 0x7f3b541f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b5ae4e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b540cac80 0x7f3b5406b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b5ae4e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b540c76a0 con 0x7f3b540c91d0 2023-07-06T01:19:33.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b5ae4e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b54158ab0 con 0x7f3b540cac80 2023-07-06T01:19:33.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b5ae4e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b540c6d00 con 0x7f3b5406d3a0 2023-07-06T01:19:33.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b58bea700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3b5406d3a0 0x7f3b54158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b58bea700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3b5406d3a0 0x7f3b54158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56848/0 (socket says 172.21.15.22:56848) 2023-07-06T01:19:33.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b58bea700 1 -- 172.21.15.22:0/1610724887 learned_addr learned my addr 172.21.15.22:0/1610724887 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:33.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b58bea700 1 -- 172.21.15.22:0/1610724887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b540cac80 msgr2=0x7f3b5406b9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T01:19:33.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b58bea700 1 --2- 172.21.15.22:0/1610724887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b540cac80 0x7f3b5406b9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b58bea700 1 -- 172.21.15.22:0/1610724887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540c91d0 msgr2=0x7f3b541f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b58bea700 1 --2- 172.21.15.22:0/1610724887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540c91d0 0x7f3b541f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f3b58bea700 1 -- 172.21.15.22:0/1610724887 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c0077e0 con 0x7f3b5406d3a0 2023-07-06T01:19:33.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7f3b58bea700 1 --2- 172.21.15.22:0/1610724887 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3b5406d3a0 0x7f3b54158570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800c520 tx=0x7f3b4800c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7f3b517fa700 1 -- 172.21.15.22:0/1610724887 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b48004c80 con 0x7f3b5406d3a0 2023-07-06T01:19:33.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7f3b517fa700 1 -- 172.21.15.22:0/1610724887 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b48004de0 con 0x7f3b5406d3a0 2023-07-06T01:19:33.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7f3b517fa700 1 -- 172.21.15.22:0/1610724887 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b480062d0 con 0x7f3b5406d3a0 2023-07-06T01:19:33.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/1610724887 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b540c6fe0 con 0x7f3b5406d3a0 2023-07-06T01:19:33.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/1610724887 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b541ff220 con 0x7f3b5406d3a0 2023-07-06T01:19:33.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7faf8c853700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf880c33f0 0x7faf880c1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7faf8c853700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf880c33f0 0x7faf880c1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:34432/0 (socket says 172.21.15.22:34432) 2023-07-06T01:19:33.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7faf8c853700 1 -- 172.21.15.22:0/2025456265 learned_addr learned my addr 172.21.15.22:0/2025456265 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:33.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7faf8c853700 1 -- 172.21.15.22:0/2025456265 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faf88165b00 msgr2=0x7faf880c5c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T01:19:33.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7faf8c853700 1 --2- 172.21.15.22:0/2025456265 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faf88165b00 0x7faf880c5c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7faf8c853700 1 -- 172.21.15.22:0/2025456265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf88165730 msgr2=0x7faf881795d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7faf8c853700 1 --2- 172.21.15.22:0/2025456265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf88165730 0x7faf881795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7faf8c853700 1 -- 172.21.15.22:0/2025456265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf880b2ee0 con 0x7faf880c33f0 2023-07-06T01:19:33.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7faf8c853700 1 --2- 172.21.15.22:0/2025456265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf880c33f0 0x7faf880c1550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf880cdfc0 tx=0x7faf80007620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.709+0000 7f7215035700 1 -- 172.21.15.22:0/1194700318 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72100874f0 con 0x7f721006d3a0 2023-07-06T01:19:33.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7f7215035700 1 -- 172.21.15.22:0/1194700318 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72100c3700 con 0x7f721006d3a0 2023-07-06T01:19:33.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.708+0000 7f6e3d15b700 1 -- 172.21.15.22:0/211695554 wait complete. 2023-07-06T01:19:33.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f6e3d15b700 1 Processor -- start 2023-07-06T01:19:33.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.710+0000 7f6e3d15b700 1 -- start start 2023-07-06T01:19:33.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7f6e3d15b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e380c9a50 0x7f6e3815b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7f6e3d15b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6e380cc750 0x7f6e38202230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7f6e3d15b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e381597a0 0x7f6e380b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7f6e3d15b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e38202770 con 0x7f6e380c9a50 2023-07-06T01:19:33.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7f6e3d15b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e380b3370 con 0x7f6e381597a0 2023-07-06T01:19:33.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7f6e3d15b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e380b3650 con 0x7f6e380cc750 2023-07-06T01:19:33.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7faf84ff9700 1 -- 172.21.15.22:0/2025456265 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8000ae40 con 0x7faf880c33f0 2023-07-06T01:19:33.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7faf84ff9700 1 -- 172.21.15.22:0/2025456265 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf80007b20 con 0x7faf880c33f0 2023-07-06T01:19:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7faf84ff9700 1 -- 172.21.15.22:0/2025456265 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80013840 con 0x7faf880c33f0 2023-07-06T01:19:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.712+0000 7f6e3659c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6e380cc750 0x7f6e38202230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.712+0000 7f6e3759e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e381597a0 0x7f6e380b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.712+0000 7f6e3659c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6e380cc750 0x7f6e38202230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56858/0 (socket says 172.21.15.22:56858) 2023-07-06T01:19:33.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7faf8e2b6700 1 -- 172.21.15.22:0/2025456265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf880b31c0 con 0x7faf880c33f0 2023-07-06T01:19:33.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.711+0000 7faf8e2b6700 1 -- 172.21.15.22:0/2025456265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf880c6190 con 0x7faf880c33f0 2023-07-06T01:19:33.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.712+0000 7f6e3659c700 1 -- 172.21.15.22:0/1183359280 learned_addr learned my addr 172.21.15.22:0/1183359280 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:33.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.713+0000 7f6e3759e700 1 -- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6e380cc750 msgr2=0x7f6e38202230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.713+0000 7f6e3759e700 1 --2- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6e380cc750 0x7f6e38202230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.713+0000 7f6e3759e700 1 -- 172.21.15.22:0/1183359280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e380c9a50 msgr2=0x7f6e3815b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.713+0000 7f6e3759e700 1 --2- 172.21.15.22:0/1183359280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e380c9a50 0x7f6e3815b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.713+0000 7f6e3759e700 1 -- 172.21.15.22:0/1183359280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3000ba90 con 0x7f6e381597a0 2023-07-06T01:19:33.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.713+0000 7f6e3659c700 1 --2- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6e380cc750 0x7f6e38202230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:33.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.712+0000 7fbbb5c48700 1 -- 172.21.15.22:0/2037761757 shutdown_connections 2023-07-06T01:19:33.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.713+0000 7fbbb5c48700 1 -- 172.21.15.22:0/2037761757 wait complete. 2023-07-06T01:19:33.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.713+0000 7faf8e2b6700 1 -- 172.21.15.22:0/2025456265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf680052f0 con 0x7faf880c33f0 2023-07-06T01:19:33.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.714+0000 7f3b517fa700 1 -- 172.21.15.22:0/1610724887 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3b48006430 con 0x7f3b5406d3a0 2023-07-06T01:19:33.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.714+0000 7faf84ff9700 1 -- 172.21.15.22:0/2025456265 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faf800139a0 con 0x7faf880c33f0 2023-07-06T01:19:33.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.714+0000 7f3b517fa700 1 --2- 172.21.15.22:0/1610724887 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3b3806ad30 0x7f3b3806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.714+0000 7f3b517fa700 1 -- 172.21.15.22:0/1610724887 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3b480885e0 con 0x7f3b5406d3a0 2023-07-06T01:19:33.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.714+0000 7f3b53fff700 1 --2- 172.21.15.22:0/1610724887 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3b3806ad30 0x7f3b3806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.714+0000 7faf84ff9700 1 --2- 172.21.15.22:0/2025456265 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7faf7406ad10 0x7faf7406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.715+0000 7faf84ff9700 1 -- 172.21.15.22:0/2025456265 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf80089420 con 0x7faf880c33f0 2023-07-06T01:19:33.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.715+0000 7fbbb5c48700 1 Processor -- start 2023-07-06T01:19:33.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.715+0000 7faf87fff700 1 --2- 172.21.15.22:0/2025456265 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7faf7406ad10 0x7faf7406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.715+0000 7f3b53fff700 1 --2- 172.21.15.22:0/1610724887 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3b3806ad30 0x7f3b3806d1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3b540cc6d0 tx=0x7f3b40006cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.715+0000 7fbbb5c48700 1 -- start start 2023-07-06T01:19:33.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.715+0000 7faf87fff700 1 --2- 172.21.15.22:0/2025456265 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7faf7406ad10 0x7faf7406d1c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf88070f90 tx=0x7faf78009800 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.716+0000 7f71f77fe700 1 -- 172.21.15.22:0/1194700318 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7204010b70 con 0x7f721006d3a0 2023-07-06T01:19:33.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.716+0000 7f286b693700 1 Processor -- start 2023-07-06T01:19:33.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.716+0000 7fbbb5c48700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb006b9e0 0x7fbbb0163730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.716+0000 7fbbb5c48700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbb006d3a0 0x7fbbb00c3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.716+0000 7fbbb5c48700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbbb0161010 0x7fbbb00af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.716+0000 7fbbb5c48700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb0163c70 con 0x7fbbb006d3a0 2023-07-06T01:19:33.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.716+0000 7fbbb5c48700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb0149d30 con 0x7fbbb006b9e0 2023-07-06T01:19:33.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.716+0000 7fbbb5c48700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00c4210 con 0x7fbbb0161010 2023-07-06T01:19:33.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.716+0000 7fbbaf7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb006b9e0 0x7fbbb0163730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7fbbaf7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb006b9e0 0x7fbbb0163730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:34452/0 (socket says 172.21.15.22:34452) 2023-07-06T01:19:33.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7fbbaf7fe700 1 -- 172.21.15.22:0/4124496774 learned_addr learned my addr 172.21.15.22:0/4124496774 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:33.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7fbbaf7fe700 1 -- 172.21.15.22:0/4124496774 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbbb0161010 msgr2=0x7fbbb00af8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T01:19:33.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7fbbaf7fe700 1 --2- 172.21.15.22:0/4124496774 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbbb0161010 0x7fbbb00af8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7fbbaf7fe700 1 -- 172.21.15.22:0/4124496774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbb006d3a0 msgr2=0x7fbbb00c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7fbbaf7fe700 1 --2- 172.21.15.22:0/4124496774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbb006d3a0 0x7fbbb00c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/1610724887 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b340052f0 con 0x7f3b5406d3a0 2023-07-06T01:19:33.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7fbbaf7fe700 1 -- 172.21.15.22:0/4124496774 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba0009550 con 0x7fbbb006b9e0 2023-07-06T01:19:33.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.716+0000 7f286b693700 1 -- start start 2023-07-06T01:19:33.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.716+0000 7f71f77fe700 1 --2- 172.21.15.22:0/1194700318 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f71fc06adb0 0x7f71fc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.716+0000 7f71f77fe700 1 -- 172.21.15.22:0/1194700318 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f720408aa50 con 0x7f721006d3a0 2023-07-06T01:19:33.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7f6e3759e700 1 --2- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e381597a0 0x7f6e380b2b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800c480 tx=0x7f6e2800c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.716+0000 7f720e59c700 1 --2- 172.21.15.22:0/1194700318 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f71fc06adb0 0x7f71fc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7fbbaf7fe700 1 --2- 172.21.15.22:0/4124496774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb006b9e0 0x7fbbb0163730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbba8001ed0 tx=0x7fbba800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7f286b693700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f286406d3a0 0x7f28640cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7f286b693700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f28640c91d0 0x7f286406d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7f286b693700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28640cb1c0 con 0x7f28640cc750 2023-07-06T01:19:33.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7f286b693700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28640cb410 con 0x7f286406d3a0 2023-07-06T01:19:33.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7f286b693700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286415d4e0 con 0x7f28640c91d0 2023-07-06T01:19:33.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.717+0000 7f720e59c700 1 --2- 172.21.15.22:0/1194700318 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f71fc06adb0 0x7f71fc06d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f72100aff80 tx=0x7f7200001980 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.718+0000 7fe181867700 1 Processor -- start 2023-07-06T01:19:33.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.718+0000 7f2869c30700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f28640cc750 0x7f28640ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38544/0 (socket says 172.21.15.22:38544) 2023-07-06T01:19:33.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.718+0000 7f2869c30700 1 -- 172.21.15.22:0/1936012090 learned_addr learned my addr 172.21.15.22:0/1936012090 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:33.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.718+0000 7f2868c2e700 1 --2- 172.21.15.22:0/1936012090 >> v2:172.21.15.103:3300/0 conn(0x7f28640c91d0 0x7f286406d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.718+0000 7fe181867700 1 -- start start 2023-07-06T01:19:33.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.718+0000 7f2868c2e700 1 -- 172.21.15.22:0/1936012090 >> v2:172.21.15.22:3300/0 conn(0x7f286406d3a0 msgr2=0x7f28640cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:33.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.718+0000 7f2868c2e700 1 --2- 172.21.15.22:0/1936012090 >> v2:172.21.15.22:3300/0 conn(0x7f286406d3a0 0x7f28640cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.718+0000 7f2868c2e700 1 -- 172.21.15.22:0/1936012090 >> v1:172.21.15.103:6789/0 conn(0x7f28640cc750 legacy=0x7f28640ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.718+0000 7f2868c2e700 1 -- 172.21.15.22:0/1936012090 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28640cdd50 con 0x7f28640c91d0 2023-07-06T01:19:33.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.718+0000 7f2868c2e700 1 --2- 172.21.15.22:0/1936012090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28640c91d0 0x7f286406d770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2864072330 tx=0x7f2860009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=320b151823001791 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7f6e1f7fe700 1 -- 172.21.15.22:0/1183359280 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28004c80 con 0x7f6e381597a0 2023-07-06T01:19:33.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7f6e3d15b700 1 -- 172.21.15.22:0/1183359280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e380b3930 con 0x7f6e381597a0 2023-07-06T01:19:33.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fbb97fff700 1 -- 172.21.15.22:0/4124496774 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba800ade0 con 0x7fbbb006b9e0 2023-07-06T01:19:33.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7faf84ff9700 1 -- 172.21.15.22:0/2025456265 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faf80054440 con 0x7faf880c33f0 2023-07-06T01:19:33.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fe181867700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe17c06d3a0 0x7fe17c0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fe181867700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe17c06d770 0x7fe17c0ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fe181867700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe17c0acdb0 0x7fe17c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fe181867700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17c0ae060 con 0x7fe17c06d3a0 2023-07-06T01:19:33.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fe181867700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17c1613a0 con 0x7fe17c0acdb0 2023-07-06T01:19:33.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fe181867700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0cf450 con 0x7fe17c06d770 2023-07-06T01:19:33.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fe17b7fe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe17c0acdb0 0x7fe17c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fe17b7fe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe17c0acdb0 0x7fe17c0ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:34464/0 (socket says 172.21.15.22:34464) 2023-07-06T01:19:33.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fe17b7fe700 1 -- 172.21.15.22:0/2217392803 learned_addr learned my addr 172.21.15.22:0/2217392803 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:33.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.720+0000 7fe17a7fc700 1 --2- 172.21.15.22:0/2217392803 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe17c06d770 0x7fe17c0ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.720+0000 7fe17b7fe700 1 -- 172.21.15.22:0/2217392803 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe17c06d770 msgr2=0x7fe17c0ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.720+0000 7fe17b7fe700 1 --2- 172.21.15.22:0/2217392803 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe17c06d770 0x7fe17c0ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.720+0000 7fe17b7fe700 1 -- 172.21.15.22:0/2217392803 >> v2:172.21.15.22:3300/0 conn(0x7fe17c06d3a0 msgr2=0x7fe17c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.720+0000 7fe17b7fe700 1 --2- 172.21.15.22:0/2217392803 >> v2:172.21.15.22:3300/0 conn(0x7fe17c06d3a0 0x7fe17c0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.720+0000 7fe17b7fe700 1 -- 172.21.15.22:0/2217392803 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c0cf730 con 0x7fe17c0acdb0 2023-07-06T01:19:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fbbb5c48700 1 -- 172.21.15.22:0/4124496774 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb00c44f0 con 0x7fbbb006b9e0 2023-07-06T01:19:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fbbb5c48700 1 -- 172.21.15.22:0/4124496774 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb0162f30 con 0x7fbbb006b9e0 2023-07-06T01:19:33.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fbb97fff700 1 -- 172.21.15.22:0/4124496774 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba80020d0 con 0x7fbbb006b9e0 2023-07-06T01:19:33.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7fbb97fff700 1 -- 172.21.15.22:0/4124496774 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8003c20 con 0x7fbbb006b9e0 2023-07-06T01:19:33.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.720+0000 7fe17b7fe700 1 --2- 172.21.15.22:0/2217392803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe17c0acdb0 0x7fe17c0ad230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe17c06afc0 tx=0x7fe170009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48325677a0cd75b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.721+0000 7f71f77fe700 1 -- 172.21.15.22:0/1194700318 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7204055d50 con 0x7f721006d3a0 2023-07-06T01:19:33.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.721+0000 7fe179ffb700 1 -- 172.21.15.22:0/2217392803 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000b870 con 0x7fe17c0acdb0 2023-07-06T01:19:33.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.721+0000 7fe179ffb700 1 -- 172.21.15.22:0/2217392803 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17000b9d0 con 0x7fe17c0acdb0 2023-07-06T01:19:33.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.721+0000 7fe179ffb700 1 -- 172.21.15.22:0/2217392803 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000bd60 con 0x7fe17c0acdb0 2023-07-06T01:19:33.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7fe181867700 1 -- 172.21.15.22:0/2217392803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe17c0acdb0 msgr2=0x7fe17c0ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7fe181867700 1 --2- 172.21.15.22:0/2217392803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe17c0acdb0 0x7fe17c0ad230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe17c06afc0 tx=0x7fe170009bd0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7f6e3d15b700 1 -- 172.21.15.22:0/1183359280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e380b3ef0 con 0x7f6e381597a0 2023-07-06T01:19:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7f6e1f7fe700 1 -- 172.21.15.22:0/1183359280 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e28004e00 con 0x7f6e381597a0 2023-07-06T01:19:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7f6e1f7fe700 1 -- 172.21.15.22:0/1183359280 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28007310 con 0x7f6e381597a0 2023-07-06T01:19:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.719+0000 7f285bfff700 1 -- 172.21.15.22:0/1936012090 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860011830 con 0x7f28640c91d0 2023-07-06T01:19:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.720+0000 7f285bfff700 1 -- 172.21.15.22:0/1936012090 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2860011990 con 0x7f28640c91d0 2023-07-06T01:19:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.720+0000 7f285bfff700 1 -- 172.21.15.22:0/1936012090 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860011d20 con 0x7f28640c91d0 2023-07-06T01:19:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.720+0000 7f286b693700 1 -- 172.21.15.22:0/1936012090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28640c91d0 msgr2=0x7f286406d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.721+0000 7f286b693700 1 --2- 172.21.15.22:0/1936012090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28640c91d0 0x7f286406d770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2864072330 tx=0x7f2860009b20 comp rx=0 tx=0).stop 2023-07-06T01:19:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.721+0000 7f286b693700 1 -- 172.21.15.22:0/1936012090 shutdown_connections 2023-07-06T01:19:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.721+0000 7f286b693700 1 --2- 172.21.15.22:0/1936012090 >> v2:172.21.15.22:3300/0 conn(0x7f286406d3a0 0x7f28640cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.721+0000 7f286b693700 1 --2- 172.21.15.22:0/1936012090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28640c91d0 0x7f286406d770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.721+0000 7f286b693700 1 -- 172.21.15.22:0/1936012090 >> 172.21.15.22:0/1936012090 conn(0x7f28640b13a0 msgr2=0x7f28640ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.721+0000 7f286b693700 1 -- 172.21.15.22:0/1936012090 shutdown_connections 2023-07-06T01:19:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.721+0000 7fbbb5c48700 1 -- 172.21.15.22:0/4124496774 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb900052f0 con 0x7fbbb006b9e0 2023-07-06T01:19:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7fbb97fff700 1 -- 172.21.15.22:0/4124496774 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbba80050d0 con 0x7fbbb006b9e0 2023-07-06T01:19:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7fe181867700 1 -- 172.21.15.22:0/2217392803 shutdown_connections 2023-07-06T01:19:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7fe181867700 1 --2- 172.21.15.22:0/2217392803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe17c0acdb0 0x7fe17c0ad230 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7fe181867700 1 --2- 172.21.15.22:0/2217392803 >> v2:172.21.15.22:3300/0 conn(0x7fe17c06d3a0 0x7fe17c0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7fe181867700 1 --2- 172.21.15.22:0/2217392803 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe17c06d770 0x7fe17c0ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7fe181867700 1 -- 172.21.15.22:0/2217392803 >> 172.21.15.22:0/2217392803 conn(0x7fe17c0b13a0 msgr2=0x7fe17c0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:33.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7f286b693700 1 -- 172.21.15.22:0/1936012090 wait complete. 2023-07-06T01:19:33.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7fbb97fff700 1 --2- 172.21.15.22:0/4124496774 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbb9c06ace0 0x7fbb9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.721+0000 7f6e3d15b700 1 -- 172.21.15.22:0/1183359280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e180052f0 con 0x7f6e381597a0 2023-07-06T01:19:33.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.721+0000 7f6e1f7fe700 1 -- 172.21.15.22:0/1183359280 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6e28007470 con 0x7f6e381597a0 2023-07-06T01:19:33.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7f6e1f7fe700 1 --2- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6e2406ace0 0x7f6e2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7f6e1f7fe700 1 -- 172.21.15.22:0/1183359280 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6e28089060 con 0x7f6e381597a0 2023-07-06T01:19:33.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7f6e36d9d700 1 --2- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6e2406ace0 0x7f6e2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.722+0000 7fbb97fff700 1 -- 172.21.15.22:0/4124496774 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbba8089650 con 0x7fbbb006b9e0 2023-07-06T01:19:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.723+0000 7fe181867700 1 -- 172.21.15.22:0/2217392803 shutdown_connections 2023-07-06T01:19:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.723+0000 7fe181867700 1 -- 172.21.15.22:0/2217392803 wait complete. 2023-07-06T01:19:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.723+0000 7f3b517fa700 1 -- 172.21.15.22:0/1610724887 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3b48053710 con 0x7f3b5406d3a0 2023-07-06T01:19:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.724+0000 7fbbaeffd700 1 --2- 172.21.15.22:0/4124496774 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbb9c06ace0 0x7fbb9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.724+0000 7f286b693700 1 Processor -- start 2023-07-06T01:19:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.724+0000 7fbbaeffd700 1 --2- 172.21.15.22:0/4124496774 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbb9c06ace0 0x7fbb9c06d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbbb01648a0 tx=0x7fbba4006cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.725+0000 7fe181867700 1 Processor -- start 2023-07-06T01:19:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.724+0000 7f286b693700 1 -- start start 2023-07-06T01:19:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.725+0000 7f286b693700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f286406d3a0 0x7f28640c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.725+0000 7f286b693700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28640c91d0 0x7f28640b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.725+0000 7f286b693700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28640cc750 0x7f286414fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.725+0000 7f286b693700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28640c6540 con 0x7f28640c91d0 2023-07-06T01:19:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.725+0000 7f286b693700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28640c5c50 con 0x7f28640cc750 2023-07-06T01:19:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.725+0000 7f286b693700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286406c8a0 con 0x7f286406d3a0 2023-07-06T01:19:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.725+0000 7f2869c30700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28640cc750 0x7f286414fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.725+0000 7f2869c30700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28640cc750 0x7f286414fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:34476/0 (socket says 172.21.15.22:34476) 2023-07-06T01:19:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.725+0000 7f2869c30700 1 -- 172.21.15.22:0/1082971141 learned_addr learned my addr 172.21.15.22:0/1082971141 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.725+0000 7f2868c2e700 1 --2- 172.21.15.22:0/1082971141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28640c91d0 0x7f28640b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7f2869c30700 1 -- 172.21.15.22:0/1082971141 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f286406d3a0 msgr2=0x7f28640c8640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T01:19:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7f2869c30700 1 --2- 172.21.15.22:0/1082971141 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f286406d3a0 0x7f28640c8640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7f2869c30700 1 -- 172.21.15.22:0/1082971141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28640c91d0 msgr2=0x7f28640b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7f2869c30700 1 --2- 172.21.15.22:0/1082971141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28640c91d0 0x7f28640b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7f2869c30700 1 -- 172.21.15.22:0/1082971141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28600094a0 con 0x7f28640cc750 2023-07-06T01:19:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7f2869c30700 1 --2- 172.21.15.22:0/1082971141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28640cc750 0x7f286414fc20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f285400c500 tx=0x7f285400c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.725+0000 7fe181867700 1 -- start start 2023-07-06T01:19:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7fe181867700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe17c06d3a0 0x7fe17c15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7fe181867700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe17c06d770 0x7fe17c14afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7fe181867700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe17c0acdb0 0x7fe17c06c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7fe181867700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c15a1a0 con 0x7fe17c06d770 2023-07-06T01:19:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7fe181867700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c14b520 con 0x7fe17c06d3a0 2023-07-06T01:19:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7fe181867700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c1592d0 con 0x7fe17c0acdb0 2023-07-06T01:19:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7fe17a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe17c06d770 0x7fe17c14afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7fe17a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe17c06d770 0x7fe17c14afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45758/0 (socket says 172.21.15.22:45758) 2023-07-06T01:19:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7fe17a7fc700 1 -- 172.21.15.22:0/3839077182 learned_addr learned my addr 172.21.15.22:0/3839077182 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.726+0000 7fe17a7fc700 1 -- 172.21.15.22:0/3839077182 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe17c0acdb0 msgr2=0x7fe17c06c020 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T01:19:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.727+0000 7fe17a7fc700 1 --2- 172.21.15.22:0/3839077182 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe17c0acdb0 0x7fe17c06c020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.727+0000 7fe17a7fc700 1 -- 172.21.15.22:0/3839077182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe17c06d3a0 msgr2=0x7fe17c15a5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T01:19:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.727+0000 7fe17a7fc700 1 --2- 172.21.15.22:0/3839077182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe17c06d3a0 0x7fe17c15a5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.727+0000 7fe17a7fc700 1 -- 172.21.15.22:0/3839077182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe170009550 con 0x7fe17c06d770 2023-07-06T01:19:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.727+0000 7fe17a7fc700 1 --2- 172.21.15.22:0/3839077182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe17c06d770 0x7fe17c14afe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe17400cd00 tx=0x7fe17400a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.727+0000 7fe163fff700 1 -- 172.21.15.22:0/3839077182 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17400acf0 con 0x7fe17c06d770 2023-07-06T01:19:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.727+0000 7fe181867700 1 -- 172.21.15.22:0/3839077182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c159520 con 0x7fe17c06d770 2023-07-06T01:19:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.727+0000 7fe163fff700 1 -- 172.21.15.22:0/3839077182 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17400ae50 con 0x7fe17c06d770 2023-07-06T01:19:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.727+0000 7fe163fff700 1 -- 172.21.15.22:0/3839077182 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174003c40 con 0x7fe17c06d770 2023-07-06T01:19:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.727+0000 7fe181867700 1 -- 172.21.15.22:0/3839077182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c159ae0 con 0x7fe17c06d770 2023-07-06T01:19:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.728+0000 7fe181867700 1 -- 172.21.15.22:0/3839077182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17c0744b0 con 0x7fe17c06d770 2023-07-06T01:19:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.727+0000 7f6e1f7fe700 1 -- 172.21.15.22:0/1183359280 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6e28054110 con 0x7f6e381597a0 2023-07-06T01:19:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.730+0000 7fe163fff700 1 -- 172.21.15.22:0/3839077182 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe174004020 con 0x7fe17c06d770 2023-07-06T01:19:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.731+0000 7fe163fff700 1 --2- 172.21.15.22:0/3839077182 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe16406ae60 0x7fe16406d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.731+0000 7fe163fff700 1 -- 172.21.15.22:0/3839077182 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe174091350 con 0x7fe17c06d770 2023-07-06T01:19:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.731+0000 7fe17affd700 1 --2- 172.21.15.22:0/3839077182 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe16406ae60 0x7fe16406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.731+0000 7fbb97fff700 1 -- 172.21.15.22:0/4124496774 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbba8054660 con 0x7fbbb006b9e0 2023-07-06T01:19:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.733+0000 7fe17affd700 1 --2- 172.21.15.22:0/3839077182 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe16406ae60 0x7fe16406d310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009690 tx=0x7fe16c006c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.740+0000 7fe163fff700 1 -- 172.21.15.22:0/3839077182 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe17408e060 con 0x7fe17c06d770 2023-07-06T01:19:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.740+0000 7f2859ffb700 1 -- 172.21.15.22:0/1082971141 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854005d70 con 0x7f28640cc750 2023-07-06T01:19:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.740+0000 7f2859ffb700 1 -- 172.21.15.22:0/1082971141 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28540070d0 con 0x7f28640cc750 2023-07-06T01:19:33.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.740+0000 7f2859ffb700 1 -- 172.21.15.22:0/1082971141 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854007440 con 0x7f28640cc750 2023-07-06T01:19:33.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.741+0000 7f286b693700 1 -- 172.21.15.22:0/1082971141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28640c5ef0 con 0x7f28640cc750 2023-07-06T01:19:33.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.741+0000 7f286b693700 1 -- 172.21.15.22:0/1082971141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286406c500 con 0x7f28640cc750 2023-07-06T01:19:33.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.742+0000 7f6e36d9d700 1 --2- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6e2406ace0 0x7f6e2406d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0097b0 tx=0x7f6e2c006d20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.742+0000 7f286b693700 1 -- 172.21.15.22:0/1082971141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28640744b0 con 0x7f28640cc750 2023-07-06T01:19:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.755+0000 7f2859ffb700 1 -- 172.21.15.22:0/1082971141 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f28540075a0 con 0x7f28640cc750 2023-07-06T01:19:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.755+0000 7f2859ffb700 1 --2- 172.21.15.22:0/1082971141 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f284806adb0 0x7f284806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.755+0000 7f2859ffb700 1 -- 172.21.15.22:0/1082971141 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2854089090 con 0x7f28640cc750 2023-07-06T01:19:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.756+0000 7f2859ffb700 1 -- 172.21.15.22:0/1082971141 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2854089510 con 0x7f28640cc750 2023-07-06T01:19:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.774+0000 7f286942f700 1 --2- 172.21.15.22:0/1082971141 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f284806adb0 0x7f284806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:33.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:33.811+0000 7f286942f700 1 --2- 172.21.15.22:0/1082971141 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f284806adb0 0x7f284806d260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f285c00ab90 tx=0x7f285c009250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:34.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.051+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2553154693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6d4c0055a0 con 0x7f6d6406d3a0 2023-07-06T01:19:34.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.052+0000 7f6d5a7fc700 1 -- 172.21.15.22:0/2553154693 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6d5c057f20 con 0x7f6d6406d3a0 2023-07-06T01:19:34.057 INFO:teuthology.orchestra.run.smithi022.stdout:55834574853 2023-07-06T01:19:34.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.055+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2553154693 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6d4806ade0 msgr2=0x7f6d4806d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.055+0000 7f6d6bb13700 1 --2- 172.21.15.22:0/2553154693 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6d4806ade0 0x7f6d4806d290 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d540010e0 tx=0x7f6d54009040 comp rx=0 tx=0).stop 2023-07-06T01:19:34.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.055+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2553154693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d6406d3a0 msgr2=0x7f6d64165010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.055+0000 7f6d6bb13700 1 --2- 172.21.15.22:0/2553154693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d6406d3a0 0x7f6d64165010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c007e40 tx=0x7f6d5c0085d0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.057+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2553154693 shutdown_connections 2023-07-06T01:19:34.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.057+0000 7f6d6bb13700 1 --2- 172.21.15.22:0/2553154693 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d64169940 0x7f6d640c3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.057+0000 7f6d6bb13700 1 --2- 172.21.15.22:0/2553154693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d6406d3a0 0x7f6d64165010 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.057+0000 7f6d6bb13700 1 --2- 172.21.15.22:0/2553154693 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6d4806ade0 0x7f6d4806d290 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.057+0000 7f6d6bb13700 1 --2- 172.21.15.22:0/2553154693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d6408a220 0x7f6d640c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.057+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2553154693 >> 172.21.15.22:0/2553154693 conn(0x7f6d640b13a0 msgr2=0x7f6d640703c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:34.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.058+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2553154693 shutdown_connections 2023-07-06T01:19:34.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.058+0000 7f6d6bb13700 1 -- 172.21.15.22:0/2553154693 wait complete. 2023-07-06T01:19:34.079 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2023-07-06T01:19:34.080 DEBUG:teuthology.parallel:result is None 2023-07-06T01:19:34.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.125+0000 7f286b693700 1 -- 172.21.15.22:0/1082971141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f28640c2450 con 0x7f28640cc750 2023-07-06T01:19:34.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.126+0000 7f2859ffb700 1 -- 172.21.15.22:0/1082971141 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f28540541d0 con 0x7f28640cc750 2023-07-06T01:19:34.129 INFO:teuthology.orchestra.run.smithi022.stdout:55834574855 2023-07-06T01:19:34.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.127+0000 7f7215035700 1 -- 172.21.15.22:0/1194700318 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f72100c3700 con 0x7f721006d3a0 2023-07-06T01:19:34.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.128+0000 7f286b693700 1 -- 172.21.15.22:0/1082971141 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f284806adb0 msgr2=0x7f284806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.128+0000 7f286b693700 1 --2- 172.21.15.22:0/1082971141 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f284806adb0 0x7f284806d260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f285c00ab90 tx=0x7f285c009250 comp rx=0 tx=0).stop 2023-07-06T01:19:34.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.128+0000 7f286b693700 1 -- 172.21.15.22:0/1082971141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28640cc750 msgr2=0x7f286414fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.128+0000 7f286b693700 1 --2- 172.21.15.22:0/1082971141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28640cc750 0x7f286414fc20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f285400c500 tx=0x7f285400c810 comp rx=0 tx=0).stop 2023-07-06T01:19:34.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.128+0000 7f286b693700 1 -- 172.21.15.22:0/1082971141 shutdown_connections 2023-07-06T01:19:34.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.128+0000 7f286b693700 1 --2- 172.21.15.22:0/1082971141 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f284806adb0 0x7f284806d260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.128+0000 7f286b693700 1 --2- 172.21.15.22:0/1082971141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28640cc750 0x7f286414fc20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.128+0000 7f286b693700 1 --2- 172.21.15.22:0/1082971141 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f286406d3a0 0x7f28640c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.128+0000 7f286b693700 1 --2- 172.21.15.22:0/1082971141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28640c91d0 0x7f28640b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.128+0000 7f286b693700 1 -- 172.21.15.22:0/1082971141 >> 172.21.15.22:0/1082971141 conn(0x7f28640b13a0 msgr2=0x7f2864156e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:34.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.128+0000 7f71f77fe700 1 -- 172.21.15.22:0/1194700318 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f72100c3700 con 0x7f721006d3a0 2023-07-06T01:19:34.133 INFO:teuthology.orchestra.run.smithi022.stdout:55834574854 2023-07-06T01:19:34.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.128+0000 7f286b693700 1 -- 172.21.15.22:0/1082971141 shutdown_connections 2023-07-06T01:19:34.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.128+0000 7f286b693700 1 -- 172.21.15.22:0/1082971141 wait complete. 2023-07-06T01:19:34.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.130+0000 7faf8e2b6700 1 -- 172.21.15.22:0/2025456265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7faf68005870 con 0x7faf880c33f0 2023-07-06T01:19:34.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.131+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/1610724887 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3b34005870 con 0x7f3b5406d3a0 2023-07-06T01:19:34.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.131+0000 7faf84ff9700 1 -- 172.21.15.22:0/2025456265 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf80057a60 con 0x7faf880c33f0 2023-07-06T01:19:34.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.131+0000 7f3b517fa700 1 -- 172.21.15.22:0/1610724887 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3b48017020 con 0x7f3b5406d3a0 2023-07-06T01:19:34.136 INFO:teuthology.orchestra.run.smithi022.stdout:60129542151 2023-07-06T01:19:34.136 INFO:teuthology.orchestra.run.smithi022.stdout:55834574854 2023-07-06T01:19:34.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.132+0000 7f7215035700 1 -- 172.21.15.22:0/1194700318 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f71fc06adb0 msgr2=0x7f71fc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.132+0000 7f7215035700 1 --2- 172.21.15.22:0/1194700318 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f71fc06adb0 0x7f71fc06d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f72100aff80 tx=0x7f7200001980 comp rx=0 tx=0).stop 2023-07-06T01:19:34.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.132+0000 7f7215035700 1 -- 172.21.15.22:0/1194700318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f721006d3a0 msgr2=0x7f7210086fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.132+0000 7f7215035700 1 --2- 172.21.15.22:0/1194700318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f721006d3a0 0x7f7210086fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7210074000 tx=0x7f7204004750 comp rx=0 tx=0).stop 2023-07-06T01:19:34.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.132+0000 7f7215035700 1 -- 172.21.15.22:0/1194700318 shutdown_connections 2023-07-06T01:19:34.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.132+0000 7f7215035700 1 --2- 172.21.15.22:0/1194700318 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7210169940 0x7f7210085680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.132+0000 7f7215035700 1 --2- 172.21.15.22:0/1194700318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f721006d3a0 0x7f7210086fb0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7210074000 tx=0x7f7204004750 comp rx=0 tx=0).stop 2023-07-06T01:19:34.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.132+0000 7f7215035700 1 --2- 172.21.15.22:0/1194700318 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f71fc06adb0 0x7f71fc06d260 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f72100aff80 tx=0x7f7200001980 comp rx=0 tx=0).stop 2023-07-06T01:19:34.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.132+0000 7f7215035700 1 --2- 172.21.15.22:0/1194700318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f721008a220 0x7f7210085140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.132+0000 7f7215035700 1 -- 172.21.15.22:0/1194700318 >> 172.21.15.22:0/1194700318 conn(0x7f72100b13a0 msgr2=0x7f7210168ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:34.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.134+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/1610724887 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3b3806ad30 msgr2=0x7f3b3806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.134+0000 7f3b5ae4e700 1 --2- 172.21.15.22:0/1610724887 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3b3806ad30 0x7f3b3806d1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3b540cc6d0 tx=0x7f3b40006cd0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.134+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/1610724887 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3b5406d3a0 msgr2=0x7f3b54158570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.134+0000 7f3b5ae4e700 1 --2- 172.21.15.22:0/1610724887 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3b5406d3a0 0x7f3b54158570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800c520 tx=0x7f3b4800c8e0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.135+0000 7faf8e2b6700 1 -- 172.21.15.22:0/2025456265 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7faf7406ad10 msgr2=0x7faf7406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.135+0000 7faf8e2b6700 1 --2- 172.21.15.22:0/2025456265 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7faf7406ad10 0x7faf7406d1c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf88070f90 tx=0x7faf78009800 comp rx=0 tx=0).stop 2023-07-06T01:19:34.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.135+0000 7faf8e2b6700 1 -- 172.21.15.22:0/2025456265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf880c33f0 msgr2=0x7faf880c1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.135+0000 7faf8e2b6700 1 --2- 172.21.15.22:0/2025456265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf880c33f0 0x7faf880c1550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf880cdfc0 tx=0x7faf80007620 comp rx=0 tx=0).stop 2023-07-06T01:19:34.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.136+0000 7f7215035700 1 -- 172.21.15.22:0/1194700318 shutdown_connections 2023-07-06T01:19:34.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.136+0000 7faf8e2b6700 1 -- 172.21.15.22:0/2025456265 shutdown_connections 2023-07-06T01:19:34.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.136+0000 7faf8e2b6700 1 --2- 172.21.15.22:0/2025456265 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7faf7406ad10 0x7faf7406d1c0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf88070f90 tx=0x7faf78009800 comp rx=0 tx=0).stop 2023-07-06T01:19:34.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.136+0000 7faf8e2b6700 1 --2- 172.21.15.22:0/2025456265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf880c33f0 0x7faf880c1550 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.136+0000 7faf8e2b6700 1 --2- 172.21.15.22:0/2025456265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf88165730 0x7faf881795d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.136+0000 7faf8e2b6700 1 --2- 172.21.15.22:0/2025456265 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faf88165b00 0x7faf880c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.136+0000 7faf8e2b6700 1 -- 172.21.15.22:0/2025456265 >> 172.21.15.22:0/2025456265 conn(0x7faf880b13a0 msgr2=0x7faf880cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:34.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.136+0000 7f7215035700 1 -- 172.21.15.22:0/1194700318 wait complete. 2023-07-06T01:19:34.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.137+0000 7f6e3d15b700 1 -- 172.21.15.22:0/1183359280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6e18005870 con 0x7f6e381597a0 2023-07-06T01:19:34.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.137+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/1610724887 shutdown_connections 2023-07-06T01:19:34.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.137+0000 7f3b5ae4e700 1 --2- 172.21.15.22:0/1610724887 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3b3806ad30 0x7f3b3806d1e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.137+0000 7f3b5ae4e700 1 --2- 172.21.15.22:0/1610724887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b540cac80 0x7f3b5406b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.137+0000 7f3b5ae4e700 1 --2- 172.21.15.22:0/1610724887 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3b5406d3a0 0x7f3b54158570 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.137+0000 7f3b5ae4e700 1 --2- 172.21.15.22:0/1610724887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b540c91d0 0x7f3b541f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.137+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/1610724887 >> 172.21.15.22:0/1610724887 conn(0x7f3b540b13a0 msgr2=0x7f3b5406d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:34.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.137+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/1610724887 shutdown_connections 2023-07-06T01:19:34.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.137+0000 7f3b5ae4e700 1 -- 172.21.15.22:0/1610724887 wait complete. 2023-07-06T01:19:34.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.137+0000 7f6e1f7fe700 1 -- 172.21.15.22:0/1183359280 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6e28057730 con 0x7f6e381597a0 2023-07-06T01:19:34.145 INFO:teuthology.orchestra.run.smithi022.stdout:55834574853 2023-07-06T01:19:34.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.141+0000 7fbbb5c48700 1 -- 172.21.15.22:0/4124496774 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbb900055a0 con 0x7fbbb006b9e0 2023-07-06T01:19:34.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.141+0000 7f6e3d15b700 1 -- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6e2406ace0 msgr2=0x7f6e2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.141+0000 7f6e3d15b700 1 --2- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6e2406ace0 0x7f6e2406d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0097b0 tx=0x7f6e2c006d20 comp rx=0 tx=0).stop 2023-07-06T01:19:34.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.141+0000 7f6e3d15b700 1 -- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e381597a0 msgr2=0x7f6e380b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.141+0000 7f6e3d15b700 1 --2- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e381597a0 0x7f6e380b2b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800c480 tx=0x7f6e2800c790 comp rx=0 tx=0).stop 2023-07-06T01:19:34.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.141+0000 7f6e3d15b700 1 -- 172.21.15.22:0/1183359280 shutdown_connections 2023-07-06T01:19:34.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.141+0000 7f6e3d15b700 1 --2- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6e2406ace0 0x7f6e2406d190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.141+0000 7f6e3d15b700 1 --2- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e381597a0 0x7f6e380b2b40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.141+0000 7f6e3d15b700 1 --2- 172.21.15.22:0/1183359280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e380c9a50 0x7f6e3815b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.141+0000 7f6e3d15b700 1 --2- 172.21.15.22:0/1183359280 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6e380cc750 0x7f6e38202230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.141+0000 7f6e3d15b700 1 -- 172.21.15.22:0/1183359280 >> 172.21.15.22:0/1183359280 conn(0x7f6e380b0710 msgr2=0x7f6e3806b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:34.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.142+0000 7faf8e2b6700 1 -- 172.21.15.22:0/2025456265 shutdown_connections 2023-07-06T01:19:34.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.142+0000 7faf8e2b6700 1 -- 172.21.15.22:0/2025456265 wait complete. 2023-07-06T01:19:34.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.142+0000 7f6e3d15b700 1 -- 172.21.15.22:0/1183359280 shutdown_connections 2023-07-06T01:19:34.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.142+0000 7f6e3d15b700 1 -- 172.21.15.22:0/1183359280 wait complete. 2023-07-06T01:19:34.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.142+0000 7fbb97fff700 1 -- 172.21.15.22:0/4124496774 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbba8057c80 con 0x7fbbb006b9e0 2023-07-06T01:19:34.151 INFO:teuthology.orchestra.run.smithi022.stdout:55834574854 2023-07-06T01:19:34.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.145+0000 7fbbb5c48700 1 -- 172.21.15.22:0/4124496774 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbb9c06ace0 msgr2=0x7fbb9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.145+0000 7fbbb5c48700 1 --2- 172.21.15.22:0/4124496774 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbb9c06ace0 0x7fbb9c06d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbbb01648a0 tx=0x7fbba4006cd0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.145+0000 7fbbb5c48700 1 -- 172.21.15.22:0/4124496774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb006b9e0 msgr2=0x7fbbb0163730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.145+0000 7fbbb5c48700 1 --2- 172.21.15.22:0/4124496774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb006b9e0 0x7fbbb0163730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbba8001ed0 tx=0x7fbba800a340 comp rx=0 tx=0).stop 2023-07-06T01:19:34.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.145+0000 7fbbb5c48700 1 -- 172.21.15.22:0/4124496774 shutdown_connections 2023-07-06T01:19:34.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.145+0000 7fbbb5c48700 1 --2- 172.21.15.22:0/4124496774 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbbb0161010 0x7fbbb00af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.145+0000 7fbbb5c48700 1 --2- 172.21.15.22:0/4124496774 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbb9c06ace0 0x7fbb9c06d190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.145+0000 7fbbb5c48700 1 --2- 172.21.15.22:0/4124496774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbb006b9e0 0x7fbbb0163730 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.145+0000 7fbbb5c48700 1 --2- 172.21.15.22:0/4124496774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbb006d3a0 0x7fbbb00c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.145+0000 7fbbb5c48700 1 -- 172.21.15.22:0/4124496774 >> 172.21.15.22:0/4124496774 conn(0x7fbbb00b13a0 msgr2=0x7fbbb00cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:34.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.146+0000 7fe181867700 1 -- 172.21.15.22:0/3839077182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe17c06e080 con 0x7fe17c06d770 2023-07-06T01:19:34.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.146+0000 7fbbb5c48700 1 -- 172.21.15.22:0/4124496774 shutdown_connections 2023-07-06T01:19:34.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.146+0000 7fbbb5c48700 1 -- 172.21.15.22:0/4124496774 wait complete. 2023-07-06T01:19:34.157 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.1 2023-07-06T01:19:34.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.146+0000 7fe163fff700 1 -- 172.21.15.22:0/3839077182 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe17408e030 con 0x7fe17c06d770 2023-07-06T01:19:34.157 INFO:teuthology.orchestra.run.smithi022.stdout:60129542151 2023-07-06T01:19:34.157 DEBUG:teuthology.parallel:result is None 2023-07-06T01:19:34.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.150+0000 7fe181867700 1 -- 172.21.15.22:0/3839077182 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe16406ae60 msgr2=0x7fe16406d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.150+0000 7fe181867700 1 --2- 172.21.15.22:0/3839077182 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe16406ae60 0x7fe16406d310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009690 tx=0x7fe16c006c10 comp rx=0 tx=0).stop 2023-07-06T01:19:34.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.150+0000 7fe181867700 1 -- 172.21.15.22:0/3839077182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe17c06d770 msgr2=0x7fe17c14afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.150+0000 7fe181867700 1 --2- 172.21.15.22:0/3839077182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe17c06d770 0x7fe17c14afe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe17400cd00 tx=0x7fe17400a4d0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.152+0000 7fe181867700 1 -- 172.21.15.22:0/3839077182 shutdown_connections 2023-07-06T01:19:34.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.152+0000 7fe181867700 1 --2- 172.21.15.22:0/3839077182 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe17c0acdb0 0x7fe17c06c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.152+0000 7fe181867700 1 --2- 172.21.15.22:0/3839077182 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe16406ae60 0x7fe16406d310 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.152+0000 7fe181867700 1 --2- 172.21.15.22:0/3839077182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe17c06d3a0 0x7fe17c15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.152+0000 7fe181867700 1 --2- 172.21.15.22:0/3839077182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe17c06d770 0x7fe17c14afe0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.152+0000 7fe181867700 1 -- 172.21.15.22:0/3839077182 >> 172.21.15.22:0/3839077182 conn(0x7fe17c0b13a0 msgr2=0x7fe17c15f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:34.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.152+0000 7fe181867700 1 -- 172.21.15.22:0/3839077182 shutdown_connections 2023-07-06T01:19:34.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.152+0000 7fe181867700 1 -- 172.21.15.22:0/3839077182 wait complete. 2023-07-06T01:19:34.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.5 2023-07-06T01:19:34.166 DEBUG:teuthology.parallel:result is None 2023-07-06T01:19:34.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.0 2023-07-06T01:19:34.167 DEBUG:teuthology.parallel:result is None 2023-07-06T01:19:34.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.4 2023-07-06T01:19:34.168 DEBUG:teuthology.parallel:result is None 2023-07-06T01:19:34.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.2 2023-07-06T01:19:34.170 DEBUG:teuthology.parallel:result is None 2023-07-06T01:19:34.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.6 2023-07-06T01:19:34.173 DEBUG:teuthology.parallel:result is None 2023-07-06T01:19:34.175 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.7 2023-07-06T01:19:34.176 DEBUG:teuthology.parallel:result is None 2023-07-06T01:19:34.176 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-06T01:19:34.176 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:19:34.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.337+0000 7fac25f99700 1 Processor -- start 2023-07-06T01:19:34.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.337+0000 7fac25f99700 1 -- start start 2023-07-06T01:19:34.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.338+0000 7fac25f99700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fac200af2a0 0x7fac2006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:34.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.338+0000 7fac25f99700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac2006a000 0x7fac2006a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:34.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.338+0000 7fac25f99700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fac200cbc10 0x7fac200cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:34.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.338+0000 7fac25f99700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2006a9e0 con 0x7fac200cbc10 2023-07-06T01:19:34.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.338+0000 7fac25f99700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac200af670 con 0x7fac200af2a0 2023-07-06T01:19:34.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.338+0000 7fac25f99700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2006d030 con 0x7fac2006a000 2023-07-06T01:19:34.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.338+0000 7fac1ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fac200cbc10 0x7fac200cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:34.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.338+0000 7fac1ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fac200cbc10 0x7fac200cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45770/0 (socket says 172.21.15.22:45770) 2023-07-06T01:19:34.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.338+0000 7fac1ffff700 1 -- 172.21.15.22:0/685163837 learned_addr learned my addr 172.21.15.22:0/685163837 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:34.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.339+0000 7fac1effd700 1 --2- 172.21.15.22:0/685163837 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac2006a000 0x7fac2006a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:34.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.339+0000 7fac1f7fe700 1 --2- 172.21.15.22:0/685163837 >> v2:172.21.15.103:3300/0 conn(0x7fac200af2a0 0x7fac2006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:34.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.339+0000 7fac1ffff700 1 -- 172.21.15.22:0/685163837 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac2006a000 msgr2=0x7fac2006a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:34.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.339+0000 7fac1ffff700 1 --2- 172.21.15.22:0/685163837 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac2006a000 0x7fac2006a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.339+0000 7fac1ffff700 1 -- 172.21.15.22:0/685163837 >> v2:172.21.15.103:3300/0 conn(0x7fac200af2a0 msgr2=0x7fac2006caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:34.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.339+0000 7fac1ffff700 1 --2- 172.21.15.22:0/685163837 >> v2:172.21.15.103:3300/0 conn(0x7fac200af2a0 0x7fac2006caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.339+0000 7fac1ffff700 1 -- 172.21.15.22:0/685163837 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac200cd560 con 0x7fac200cbc10 2023-07-06T01:19:34.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.339+0000 7fac1effd700 1 --2- 172.21.15.22:0/685163837 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac2006a000 0x7fac2006a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.339+0000 7fac1ffff700 1 --2- 172.21.15.22:0/685163837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac200cbc10 0x7fac200cc090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fac20067460 tx=0x7fac1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e911019292bc2d0d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.339+0000 7fac1e7fc700 1 -- 172.21.15.22:0/685163837 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1800e070 con 0x7fac200cbc10 2023-07-06T01:19:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.339+0000 7fac1e7fc700 1 -- 172.21.15.22:0/685163837 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1800b980 con 0x7fac200cbc10 2023-07-06T01:19:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.339+0000 7fac1e7fc700 1 -- 172.21.15.22:0/685163837 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1800bd30 con 0x7fac200cbc10 2023-07-06T01:19:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.340+0000 7fac25f99700 1 -- 172.21.15.22:0/685163837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac200cbc10 msgr2=0x7fac200cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.340+0000 7fac25f99700 1 --2- 172.21.15.22:0/685163837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac200cbc10 0x7fac200cc090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fac20067460 tx=0x7fac1800b330 comp rx=0 tx=0).stop 2023-07-06T01:19:34.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.340+0000 7fac25f99700 1 -- 172.21.15.22:0/685163837 shutdown_connections 2023-07-06T01:19:34.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.340+0000 7fac25f99700 1 --2- 172.21.15.22:0/685163837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac200cbc10 0x7fac200cc090 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.340+0000 7fac25f99700 1 --2- 172.21.15.22:0/685163837 >> v2:172.21.15.103:3300/0 conn(0x7fac200af2a0 0x7fac2006caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.340+0000 7fac25f99700 1 --2- 172.21.15.22:0/685163837 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac2006a000 0x7fac2006a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.340+0000 7fac25f99700 1 -- 172.21.15.22:0/685163837 >> 172.21.15.22:0/685163837 conn(0x7fac200b1890 msgr2=0x7fac200b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:34.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.340+0000 7fac25f99700 1 -- 172.21.15.22:0/685163837 shutdown_connections 2023-07-06T01:19:34.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.340+0000 7fac25f99700 1 -- 172.21.15.22:0/685163837 wait complete. 2023-07-06T01:19:34.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.341+0000 7fac25f99700 1 Processor -- start 2023-07-06T01:19:34.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.341+0000 7fac25f99700 1 -- start start 2023-07-06T01:19:34.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac25f99700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac2006a000 0x7fac201594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:34.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac25f99700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac200af2a0 0x7fac200adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:34.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac25f99700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac200cbc10 0x7fac200ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:34.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac25f99700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2015efc0 con 0x7fac200cbc10 2023-07-06T01:19:34.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac25f99700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2015f2a0 con 0x7fac200af2a0 2023-07-06T01:19:34.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac25f99700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac20159a10 con 0x7fac2006a000 2023-07-06T01:19:34.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac1ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac200cbc10 0x7fac200ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:34.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac1ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac200cbc10 0x7fac200ae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45776/0 (socket says 172.21.15.22:45776) 2023-07-06T01:19:34.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac1ffff700 1 -- 172.21.15.22:0/3271762159 learned_addr learned my addr 172.21.15.22:0/3271762159 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:34.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac1effd700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac200af2a0 0x7fac200adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:34.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac1f7fe700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac2006a000 0x7fac201594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:34.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac1ffff700 1 -- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac2006a000 msgr2=0x7fac201594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac1ffff700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac2006a000 0x7fac201594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac1ffff700 1 -- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac200af2a0 msgr2=0x7fac200adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac1ffff700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac200af2a0 0x7fac200adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac1ffff700 1 -- 172.21.15.22:0/3271762159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac18009b30 con 0x7fac200cbc10 2023-07-06T01:19:34.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.342+0000 7fac1ffff700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac200cbc10 0x7fac200ae1e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fac1800b730 tx=0x7fac18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:34.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.343+0000 7fac17fff700 1 -- 172.21.15.22:0/3271762159 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18004450 con 0x7fac200cbc10 2023-07-06T01:19:34.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.343+0000 7fac17fff700 1 -- 172.21.15.22:0/3271762159 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac180045b0 con 0x7fac200cbc10 2023-07-06T01:19:34.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.343+0000 7fac25f99700 1 -- 172.21.15.22:0/3271762159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2015e340 con 0x7fac200cbc10 2023-07-06T01:19:34.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.343+0000 7fac17fff700 1 -- 172.21.15.22:0/3271762159 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18004960 con 0x7fac200cbc10 2023-07-06T01:19:34.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.343+0000 7fac25f99700 1 -- 172.21.15.22:0/3271762159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2015e900 con 0x7fac200cbc10 2023-07-06T01:19:34.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.344+0000 7fac25f99700 1 -- 172.21.15.22:0/3271762159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2015ca90 con 0x7fac200cbc10 2023-07-06T01:19:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.344+0000 7fac17fff700 1 -- 172.21.15.22:0/3271762159 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fac1801b430 con 0x7fac200cbc10 2023-07-06T01:19:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.344+0000 7fac17fff700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fac0006ace0 0x7fac0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.344+0000 7fac17fff700 1 -- 172.21.15.22:0/3271762159 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fac1808a7d0 con 0x7fac200cbc10 2023-07-06T01:19:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.345+0000 7fac1f7fe700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fac0006ace0 0x7fac0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.345+0000 7fac1f7fe700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fac0006ace0 0x7fac0006d190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fac08006b20 tx=0x7fac08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:34.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.349+0000 7fac17fff700 1 -- 172.21.15.22:0/3271762159 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fac2015ca90 con 0x7fac200cbc10 2023-07-06T01:19:34.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.544+0000 7fac25f99700 1 -- 172.21.15.22:0/3271762159 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac200cc9e0 con 0x7fac0006ace0 2023-07-06T01:19:34.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.553+0000 7fac17fff700 1 -- 172.21.15.22:0/3271762159 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286822 (secure 0 0 0) 0x7fac08009dc0 con 0x7fac0006ace0 2023-07-06T01:19:34.559 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:34.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.556+0000 7fac25f99700 1 -- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fac0006ace0 msgr2=0x7fac0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.556+0000 7fac25f99700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fac0006ace0 0x7fac0006d190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fac08006b20 tx=0x7fac08008040 comp rx=0 tx=0).stop 2023-07-06T01:19:34.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.556+0000 7fac25f99700 1 -- 172.21.15.22:0/3271762159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac200cbc10 msgr2=0x7fac200ae1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.556+0000 7fac25f99700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac200cbc10 0x7fac200ae1e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fac1800b730 tx=0x7fac18004ad0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.556+0000 7fac25f99700 1 -- 172.21.15.22:0/3271762159 shutdown_connections 2023-07-06T01:19:34.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.556+0000 7fac25f99700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fac200cbc10 0x7fac200ae1e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.556+0000 7fac25f99700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac2006a000 0x7fac201594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.556+0000 7fac25f99700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fac0006ace0 0x7fac0006d190 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.556+0000 7fac25f99700 1 --2- 172.21.15.22:0/3271762159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac200af2a0 0x7fac200adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.556+0000 7fac25f99700 1 -- 172.21.15.22:0/3271762159 >> 172.21.15.22:0/3271762159 conn(0x7fac200b1890 msgr2=0x7fac200aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:34.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.556+0000 7fac25f99700 1 -- 172.21.15.22:0/3271762159 shutdown_connections 2023-07-06T01:19:34.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.556+0000 7fac25f99700 1 -- 172.21.15.22:0/3271762159 wait complete. 2023-07-06T01:19:34.563 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:19:34.571 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-07-06T01:19:33.999628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218504,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756216,"statfs":{"total":773094113280,"available":772870365184,"internally_reserved":0,"allocated":2957312,"data_stored":1813772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.369917"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068502+0000","last_change":"2023-07-06T01:19:25.653405+0000","last_active":"2023-07-06T01:19:27.068502+0000","last_peered":"2023-07-06T01:19:27.068502+0000","last_clean":"2023-07-06T01:19:27.068502+0000","last_became_active":"2023-07-06T01:19:25.653093+0000","last_became_peered":"2023-07-06T01:19:25.653093+0000","last_unstale":"2023-07-06T01:19:27.068502+0000","last_undegraded":"2023-07-06T01:19:27.068502+0000","last_fullsized":"2023-07-06T01:19:27.068502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:17.116328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632301+0000","last_change":"2023-07-06T01:19:25.649747+0000","last_active":"2023-07-06T01:19:26.632301+0000","last_peered":"2023-07-06T01:19:26.632301+0000","last_clean":"2023-07-06T01:19:26.632301+0000","last_became_active":"2023-07-06T01:19:25.649468+0000","last_became_peered":"2023-07-06T01:19:25.649468+0000","last_unstale":"2023-07-06T01:19:26.632301+0000","last_undegraded":"2023-07-06T01:19:26.632301+0000","last_fullsized":"2023-07-06T01:19:26.632301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:00.621635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632173+0000","last_change":"2023-07-06T01:19:25.660551+0000","last_active":"2023-07-06T01:19:26.632173+0000","last_peered":"2023-07-06T01:19:26.632173+0000","last_clean":"2023-07-06T01:19:26.632173+0000","last_became_active":"2023-07-06T01:19:25.660278+0000","last_became_peered":"2023-07-06T01:19:25.660278+0000","last_unstale":"2023-07-06T01:19:26.632173+0000","last_undegraded":"2023-07-06T01:19:26.632173+0000","last_fullsized":"2023-07-06T01:19:26.632173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:36.632721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068877+0000","last_change":"2023-07-06T01:19:25.647264+0000","last_active":"2023-07-06T01:19:27.068877+0000","last_peered":"2023-07-06T01:19:27.068877+0000","last_clean":"2023-07-06T01:19:27.068877+0000","last_became_active":"2023-07-06T01:19:25.646962+0000","last_became_peered":"2023-07-06T01:19:25.646962+0000","last_unstale":"2023-07-06T01:19:27.068877+0000","last_undegraded":"2023-07-06T01:19:27.068877+0000","last_fullsized":"2023-07-06T01:19:27.068877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:33.168360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631056+0000","last_change":"2023-07-06T01:19:25.655455+0000","last_active":"2023-07-06T01:19:26.631056+0000","last_peered":"2023-07-06T01:19:26.631056+0000","last_clean":"2023-07-06T01:19:26.631056+0000","last_became_active":"2023-07-06T01:19:25.655137+0000","last_became_peered":"2023-07-06T01:19:25.655137+0000","last_unstale":"2023-07-06T01:19:26.631056+0000","last_undegraded":"2023-07-06T01:19:26.631056+0000","last_fullsized":"2023-07-06T01:19:26.631056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:39.980297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631681+0000","last_change":"2023-07-06T01:19:25.660076+0000","last_active":"2023-07-06T01:19:26.631681+0000","last_peered":"2023-07-06T01:19:26.631681+0000","last_clean":"2023-07-06T01:19:26.631681+0000","last_became_active":"2023-07-06T01:19:25.659798+0000","last_became_peered":"2023-07-06T01:19:25.659798+0000","last_unstale":"2023-07-06T01:19:26.631681+0000","last_undegraded":"2023-07-06T01:19:26.631681+0000","last_fullsized":"2023-07-06T01:19:26.631681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:25.724682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068900+0000","last_change":"2023-07-06T01:19:25.649766+0000","last_active":"2023-07-06T01:19:27.068900+0000","last_peered":"2023-07-06T01:19:27.068900+0000","last_clean":"2023-07-06T01:19:27.068900+0000","last_became_active":"2023-07-06T01:19:25.649497+0000","last_became_peered":"2023-07-06T01:19:25.649497+0000","last_unstale":"2023-07-06T01:19:27.068900+0000","last_undegraded":"2023-07-06T01:19:27.068900+0000","last_fullsized":"2023-07-06T01:19:27.068900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:25.724682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631105+0000","last_change":"2023-07-06T01:19:25.657379+0000","last_active":"2023-07-06T01:19:26.631105+0000","last_peered":"2023-07-06T01:19:26.631105+0000","last_clean":"2023-07-06T01:19:26.631105+0000","last_became_active":"2023-07-06T01:19:25.657044+0000","last_became_peered":"2023-07-06T01:19:25.657044+0000","last_unstale":"2023-07-06T01:19:26.631105+0000","last_undegraded":"2023-07-06T01:19:26.631105+0000","last_fullsized":"2023-07-06T01:19:26.631105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:21:00.969228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630543+0000","last_change":"2023-07-06T01:19:25.654446+0000","last_active":"2023-07-06T01:19:26.630543+0000","last_peered":"2023-07-06T01:19:26.630543+0000","last_clean":"2023-07-06T01:19:26.630543+0000","last_became_active":"2023-07-06T01:19:25.653834+0000","last_became_peered":"2023-07-06T01:19:25.653834+0000","last_unstale":"2023-07-06T01:19:26.630543+0000","last_undegraded":"2023-07-06T01:19:26.630543+0000","last_fullsized":"2023-07-06T01:19:26.630543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:33.168360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632483+0000","last_change":"2023-07-06T01:19:25.648054+0000","last_active":"2023-07-06T01:19:26.632483+0000","last_peered":"2023-07-06T01:19:26.632483+0000","last_clean":"2023-07-06T01:19:26.632483+0000","last_became_active":"2023-07-06T01:19:25.647795+0000","last_became_peered":"2023-07-06T01:19:25.647795+0000","last_unstale":"2023-07-06T01:19:26.632483+0000","last_undegraded":"2023-07-06T01:19:26.632483+0000","last_fullsized":"2023-07-06T01:19:26.632483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632291+0000","last_change":"2023-07-06T01:19:25.657523+0000","last_active":"2023-07-06T01:19:26.632291+0000","last_peered":"2023-07-06T01:19:26.632291+0000","last_clean":"2023-07-06T01:19:26.632291+0000","last_became_active":"2023-07-06T01:19:25.657258+0000","last_became_peered":"2023-07-06T01:19:25.657258+0000","last_unstale":"2023-07-06T01:19:26.632291+0000","last_undegraded":"2023-07-06T01:19:26.632291+0000","last_fullsized":"2023-07-06T01:19:26.632291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:59.279465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068838+0000","last_change":"2023-07-06T01:19:25.660991+0000","last_active":"2023-07-06T01:19:27.068838+0000","last_peered":"2023-07-06T01:19:27.068838+0000","last_clean":"2023-07-06T01:19:27.068838+0000","last_became_active":"2023-07-06T01:19:25.660654+0000","last_became_peered":"2023-07-06T01:19:25.660654+0000","last_unstale":"2023-07-06T01:19:27.068838+0000","last_undegraded":"2023-07-06T01:19:27.068838+0000","last_fullsized":"2023-07-06T01:19:27.068838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:43:26.720400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068325+0000","last_change":"2023-07-06T01:19:25.654120+0000","last_active":"2023-07-06T01:19:27.068325+0000","last_peered":"2023-07-06T01:19:27.068325+0000","last_clean":"2023-07-06T01:19:27.068325+0000","last_became_active":"2023-07-06T01:19:25.653844+0000","last_became_peered":"2023-07-06T01:19:25.653844+0000","last_unstale":"2023-07-06T01:19:27.068325+0000","last_undegraded":"2023-07-06T01:19:27.068325+0000","last_fullsized":"2023-07-06T01:19:27.068325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:42:37.007207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631522+0000","last_change":"2023-07-06T01:19:25.650622+0000","last_active":"2023-07-06T01:19:26.631522+0000","last_peered":"2023-07-06T01:19:26.631522+0000","last_clean":"2023-07-06T01:19:26.631522+0000","last_became_active":"2023-07-06T01:19:25.650310+0000","last_became_peered":"2023-07-06T01:19:25.650310+0000","last_unstale":"2023-07-06T01:19:26.631522+0000","last_undegraded":"2023-07-06T01:19:26.631522+0000","last_fullsized":"2023-07-06T01:19:26.631522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:21.566043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069099+0000","last_change":"2023-07-06T01:19:25.651765+0000","last_active":"2023-07-06T01:19:27.069099+0000","last_peered":"2023-07-06T01:19:27.069099+0000","last_clean":"2023-07-06T01:19:27.069099+0000","last_became_active":"2023-07-06T01:19:25.651244+0000","last_became_peered":"2023-07-06T01:19:25.651244+0000","last_unstale":"2023-07-06T01:19:27.069099+0000","last_undegraded":"2023-07-06T01:19:27.069099+0000","last_fullsized":"2023-07-06T01:19:27.069099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:41:34.533716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631383+0000","last_change":"2023-07-06T01:19:25.663299+0000","last_active":"2023-07-06T01:19:26.631383+0000","last_peered":"2023-07-06T01:19:26.631383+0000","last_clean":"2023-07-06T01:19:26.631383+0000","last_became_active":"2023-07-06T01:19:25.663032+0000","last_became_peered":"2023-07-06T01:19:25.663032+0000","last_unstale":"2023-07-06T01:19:26.631383+0000","last_undegraded":"2023-07-06T01:19:26.631383+0000","last_fullsized":"2023-07-06T01:19:26.631383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:31.571390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630591+0000","last_change":"2023-07-06T01:19:25.654669+0000","last_active":"2023-07-06T01:19:26.630591+0000","last_peered":"2023-07-06T01:19:26.630591+0000","last_clean":"2023-07-06T01:19:26.630591+0000","last_became_active":"2023-07-06T01:19:25.654161+0000","last_became_peered":"2023-07-06T01:19:25.654161+0000","last_unstale":"2023-07-06T01:19:26.630591+0000","last_undegraded":"2023-07-06T01:19:26.630591+0000","last_fullsized":"2023-07-06T01:19:26.630591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:42.802038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631594+0000","last_change":"2023-07-06T01:19:25.651140+0000","last_active":"2023-07-06T01:19:26.631594+0000","last_peered":"2023-07-06T01:19:26.631594+0000","last_clean":"2023-07-06T01:19:26.631594+0000","last_became_active":"2023-07-06T01:19:25.650810+0000","last_became_peered":"2023-07-06T01:19:25.650810+0000","last_unstale":"2023-07-06T01:19:26.631594+0000","last_undegraded":"2023-07-06T01:19:26.631594+0000","last_fullsized":"2023-07-06T01:19:26.631594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:39.980297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068611+0000","last_change":"2023-07-06T01:19:25.645030+0000","last_active":"2023-07-06T01:19:27.068611+0000","last_peered":"2023-07-06T01:19:27.068611+0000","last_clean":"2023-07-06T01:19:27.068611+0000","last_became_active":"2023-07-06T01:19:25.644689+0000","last_became_peered":"2023-07-06T01:19:25.644689+0000","last_unstale":"2023-07-06T01:19:27.068611+0000","last_undegraded":"2023-07-06T01:19:27.068611+0000","last_fullsized":"2023-07-06T01:19:27.068611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:14.823559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632289+0000","last_change":"2023-07-06T01:19:25.660308+0000","last_active":"2023-07-06T01:19:26.632289+0000","last_peered":"2023-07-06T01:19:26.632289+0000","last_clean":"2023-07-06T01:19:26.632289+0000","last_became_active":"2023-07-06T01:19:25.658245+0000","last_became_peered":"2023-07-06T01:19:25.658245+0000","last_unstale":"2023-07-06T01:19:26.632289+0000","last_undegraded":"2023-07-06T01:19:26.632289+0000","last_fullsized":"2023-07-06T01:19:26.632289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:14.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069296+0000","last_change":"2023-07-06T01:19:25.660004+0000","last_active":"2023-07-06T01:19:27.069296+0000","last_peered":"2023-07-06T01:19:27.069296+0000","last_clean":"2023-07-06T01:19:27.069296+0000","last_became_active":"2023-07-06T01:19:25.659487+0000","last_became_peered":"2023-07-06T01:19:25.659487+0000","last_unstale":"2023-07-06T01:19:27.069296+0000","last_undegraded":"2023-07-06T01:19:27.069296+0000","last_fullsized":"2023-07-06T01:19:27.069296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:14.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630591+0000","last_change":"2023-07-06T01:19:25.636207+0000","last_active":"2023-07-06T01:19:26.630591+0000","last_peered":"2023-07-06T01:19:26.630591+0000","last_clean":"2023-07-06T01:19:26.630591+0000","last_became_active":"2023-07-06T01:19:25.635876+0000","last_became_peered":"2023-07-06T01:19:25.635876+0000","last_unstale":"2023-07-06T01:19:26.630591+0000","last_undegraded":"2023-07-06T01:19:26.630591+0000","last_fullsized":"2023-07-06T01:19:26.630591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:25.693365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944279+0000","last_change":"2023-07-06T01:19:25.655490+0000","last_active":"2023-07-06T01:19:26.944279+0000","last_peered":"2023-07-06T01:19:26.944279+0000","last_clean":"2023-07-06T01:19:26.944279+0000","last_became_active":"2023-07-06T01:19:25.655137+0000","last_became_peered":"2023-07-06T01:19:25.655137+0000","last_unstale":"2023-07-06T01:19:26.944279+0000","last_undegraded":"2023-07-06T01:19:26.944279+0000","last_fullsized":"2023-07-06T01:19:26.944279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:09.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068827+0000","last_change":"2023-07-06T01:19:25.655082+0000","last_active":"2023-07-06T01:19:27.068827+0000","last_peered":"2023-07-06T01:19:27.068827+0000","last_clean":"2023-07-06T01:19:27.068827+0000","last_became_active":"2023-07-06T01:19:25.654779+0000","last_became_peered":"2023-07-06T01:19:25.654779+0000","last_unstale":"2023-07-06T01:19:27.068827+0000","last_undegraded":"2023-07-06T01:19:27.068827+0000","last_fullsized":"2023-07-06T01:19:27.068827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:22.147924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068831+0000","last_change":"2023-07-06T01:19:25.644786+0000","last_active":"2023-07-06T01:19:27.068831+0000","last_peered":"2023-07-06T01:19:27.068831+0000","last_clean":"2023-07-06T01:19:27.068831+0000","last_became_active":"2023-07-06T01:19:25.644482+0000","last_became_peered":"2023-07-06T01:19:25.644482+0000","last_unstale":"2023-07-06T01:19:27.068831+0000","last_undegraded":"2023-07-06T01:19:27.068831+0000","last_fullsized":"2023-07-06T01:19:27.068831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:42.802038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068413+0000","last_change":"2023-07-06T01:19:25.642117+0000","last_active":"2023-07-06T01:19:27.068413+0000","last_peered":"2023-07-06T01:19:27.068413+0000","last_clean":"2023-07-06T01:19:27.068413+0000","last_became_active":"2023-07-06T01:19:25.641830+0000","last_became_peered":"2023-07-06T01:19:25.641830+0000","last_unstale":"2023-07-06T01:19:27.068413+0000","last_undegraded":"2023-07-06T01:19:27.068413+0000","last_fullsized":"2023-07-06T01:19:27.068413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:43:11.310891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631861+0000","last_change":"2023-07-06T01:19:25.656605+0000","last_active":"2023-07-06T01:19:26.631861+0000","last_peered":"2023-07-06T01:19:26.631861+0000","last_clean":"2023-07-06T01:19:26.631861+0000","last_became_active":"2023-07-06T01:19:25.656259+0000","last_became_peered":"2023-07-06T01:19:25.656259+0000","last_unstale":"2023-07-06T01:19:26.631861+0000","last_undegraded":"2023-07-06T01:19:26.631861+0000","last_fullsized":"2023-07-06T01:19:26.631861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:40.213207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631966+0000","last_change":"2023-07-06T01:19:24.646013+0000","last_active":"2023-07-06T01:19:26.631966+0000","last_peered":"2023-07-06T01:19:26.631966+0000","last_clean":"2023-07-06T01:19:26.631966+0000","last_became_active":"2023-07-06T01:19:24.644677+0000","last_became_peered":"2023-07-06T01:19:24.644677+0000","last_unstale":"2023-07-06T01:19:26.631966+0000","last_undegraded":"2023-07-06T01:19:26.631966+0000","last_fullsized":"2023-07-06T01:19:26.631966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:17.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944791+0000","last_change":"2023-07-06T01:19:25.650690+0000","last_active":"2023-07-06T01:19:26.944791+0000","last_peered":"2023-07-06T01:19:26.944791+0000","last_clean":"2023-07-06T01:19:26.944791+0000","last_became_active":"2023-07-06T01:19:25.650316+0000","last_became_peered":"2023-07-06T01:19:25.650316+0000","last_unstale":"2023-07-06T01:19:26.944791+0000","last_undegraded":"2023-07-06T01:19:26.944791+0000","last_fullsized":"2023-07-06T01:19:26.944791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:02.031653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068547+0000","last_change":"2023-07-06T01:19:24.645053+0000","last_active":"2023-07-06T01:19:27.068547+0000","last_peered":"2023-07-06T01:19:27.068547+0000","last_clean":"2023-07-06T01:19:27.068547+0000","last_became_active":"2023-07-06T01:19:24.644766+0000","last_became_peered":"2023-07-06T01:19:24.644766+0000","last_unstale":"2023-07-06T01:19:27.068547+0000","last_undegraded":"2023-07-06T01:19:27.068547+0000","last_fullsized":"2023-07-06T01:19:27.068547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:43.286430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630830+0000","last_change":"2023-07-06T01:19:25.649424+0000","last_active":"2023-07-06T01:19:26.630830+0000","last_peered":"2023-07-06T01:19:26.630830+0000","last_clean":"2023-07-06T01:19:26.630830+0000","last_became_active":"2023-07-06T01:19:25.649164+0000","last_became_peered":"2023-07-06T01:19:25.649164+0000","last_unstale":"2023-07-06T01:19:26.630830+0000","last_undegraded":"2023-07-06T01:19:26.630830+0000","last_fullsized":"2023-07-06T01:19:26.630830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:00.621635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944431+0000","last_change":"2023-07-06T01:19:24.645861+0000","last_active":"2023-07-06T01:19:26.944431+0000","last_peered":"2023-07-06T01:19:26.944431+0000","last_clean":"2023-07-06T01:19:26.944431+0000","last_became_active":"2023-07-06T01:19:24.645583+0000","last_became_peered":"2023-07-06T01:19:24.645583+0000","last_unstale":"2023-07-06T01:19:26.944431+0000","last_undegraded":"2023-07-06T01:19:26.944431+0000","last_fullsized":"2023-07-06T01:19:26.944431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:38.975082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069581+0000","last_change":"2023-07-06T01:19:25.656345+0000","last_active":"2023-07-06T01:19:27.069581+0000","last_peered":"2023-07-06T01:19:27.069581+0000","last_clean":"2023-07-06T01:19:27.069581+0000","last_became_active":"2023-07-06T01:19:25.655955+0000","last_became_peered":"2023-07-06T01:19:25.655955+0000","last_unstale":"2023-07-06T01:19:27.069581+0000","last_undegraded":"2023-07-06T01:19:27.069581+0000","last_fullsized":"2023-07-06T01:19:27.069581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:45.855998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630810+0000","last_change":"2023-07-06T01:19:24.643151+0000","last_active":"2023-07-06T01:19:26.630810+0000","last_peered":"2023-07-06T01:19:26.630810+0000","last_clean":"2023-07-06T01:19:26.630810+0000","last_became_active":"2023-07-06T01:19:24.642827+0000","last_became_peered":"2023-07-06T01:19:24.642827+0000","last_unstale":"2023-07-06T01:19:26.630810+0000","last_undegraded":"2023-07-06T01:19:26.630810+0000","last_fullsized":"2023-07-06T01:19:26.630810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:08.965590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944402+0000","last_change":"2023-07-06T01:19:25.652005+0000","last_active":"2023-07-06T01:19:26.944402+0000","last_peered":"2023-07-06T01:19:26.944402+0000","last_clean":"2023-07-06T01:19:26.944402+0000","last_became_active":"2023-07-06T01:19:25.651299+0000","last_became_peered":"2023-07-06T01:19:25.651299+0000","last_unstale":"2023-07-06T01:19:26.944402+0000","last_undegraded":"2023-07-06T01:19:26.944402+0000","last_fullsized":"2023-07-06T01:19:26.944402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632387+0000","last_change":"2023-07-06T01:19:24.642960+0000","last_active":"2023-07-06T01:19:26.632387+0000","last_peered":"2023-07-06T01:19:26.632387+0000","last_clean":"2023-07-06T01:19:26.632387+0000","last_became_active":"2023-07-06T01:19:24.642667+0000","last_became_peered":"2023-07-06T01:19:24.642667+0000","last_unstale":"2023-07-06T01:19:26.632387+0000","last_undegraded":"2023-07-06T01:19:26.632387+0000","last_fullsized":"2023-07-06T01:19:26.632387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:30.566175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944596+0000","last_change":"2023-07-06T01:19:25.650426+0000","last_active":"2023-07-06T01:19:26.944596+0000","last_peered":"2023-07-06T01:19:26.944596+0000","last_clean":"2023-07-06T01:19:26.944596+0000","last_became_active":"2023-07-06T01:19:25.650130+0000","last_became_peered":"2023-07-06T01:19:25.650130+0000","last_unstale":"2023-07-06T01:19:26.944596+0000","last_undegraded":"2023-07-06T01:19:26.944596+0000","last_fullsized":"2023-07-06T01:19:26.944596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:45:08.937322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069529+0000","last_change":"2023-07-06T01:19:24.644753+0000","last_active":"2023-07-06T01:19:27.069529+0000","last_peered":"2023-07-06T01:19:27.069529+0000","last_clean":"2023-07-06T01:19:27.069529+0000","last_became_active":"2023-07-06T01:19:24.644420+0000","last_became_peered":"2023-07-06T01:19:24.644420+0000","last_unstale":"2023-07-06T01:19:27.069529+0000","last_undegraded":"2023-07-06T01:19:27.069529+0000","last_fullsized":"2023-07-06T01:19:27.069529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:01.026438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068483+0000","last_change":"2023-07-06T01:19:25.652623+0000","last_active":"2023-07-06T01:19:27.068483+0000","last_peered":"2023-07-06T01:19:27.068483+0000","last_clean":"2023-07-06T01:19:27.068483+0000","last_became_active":"2023-07-06T01:19:25.652341+0000","last_became_peered":"2023-07-06T01:19:25.652341+0000","last_unstale":"2023-07-06T01:19:27.068483+0000","last_undegraded":"2023-07-06T01:19:27.068483+0000","last_fullsized":"2023-07-06T01:19:27.068483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:21:00.969228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632583+0000","last_change":"2023-07-06T01:19:24.648863+0000","last_active":"2023-07-06T01:19:26.632583+0000","last_peered":"2023-07-06T01:19:26.632583+0000","last_clean":"2023-07-06T01:19:26.632583+0000","last_became_active":"2023-07-06T01:19:24.648520+0000","last_became_peered":"2023-07-06T01:19:24.648520+0000","last_unstale":"2023-07-06T01:19:26.632583+0000","last_undegraded":"2023-07-06T01:19:26.632583+0000","last_fullsized":"2023-07-06T01:19:26.632583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:31:21.451680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632381+0000","last_change":"2023-07-06T01:19:25.653993+0000","last_active":"2023-07-06T01:19:26.632381+0000","last_peered":"2023-07-06T01:19:26.632381+0000","last_clean":"2023-07-06T01:19:26.632381+0000","last_became_active":"2023-07-06T01:19:25.653093+0000","last_became_peered":"2023-07-06T01:19:25.653093+0000","last_unstale":"2023-07-06T01:19:26.632381+0000","last_undegraded":"2023-07-06T01:19:26.632381+0000","last_fullsized":"2023-07-06T01:19:26.632381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:02.031653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069278+0000","last_change":"2023-07-06T01:19:24.651048+0000","last_active":"2023-07-06T01:19:27.069278+0000","last_peered":"2023-07-06T01:19:27.069278+0000","last_clean":"2023-07-06T01:19:27.069278+0000","last_became_active":"2023-07-06T01:19:24.650735+0000","last_became_peered":"2023-07-06T01:19:24.650735+0000","last_unstale":"2023-07-06T01:19:27.069278+0000","last_undegraded":"2023-07-06T01:19:27.069278+0000","last_fullsized":"2023-07-06T01:19:27.069278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:58.274250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944680+0000","last_change":"2023-07-06T01:19:25.651034+0000","last_active":"2023-07-06T01:19:26.944680+0000","last_peered":"2023-07-06T01:19:26.944680+0000","last_clean":"2023-07-06T01:19:26.944680+0000","last_became_active":"2023-07-06T01:19:25.650753+0000","last_became_peered":"2023-07-06T01:19:25.650753+0000","last_unstale":"2023-07-06T01:19:26.944680+0000","last_undegraded":"2023-07-06T01:19:26.944680+0000","last_fullsized":"2023-07-06T01:19:26.944680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:36.632721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630793+0000","last_change":"2023-07-06T01:19:24.648974+0000","last_active":"2023-07-06T01:19:26.630793+0000","last_peered":"2023-07-06T01:19:26.630793+0000","last_clean":"2023-07-06T01:19:26.630793+0000","last_became_active":"2023-07-06T01:19:24.648691+0000","last_became_peered":"2023-07-06T01:19:24.648691+0000","last_unstale":"2023-07-06T01:19:26.630793+0000","last_undegraded":"2023-07-06T01:19:26.630793+0000","last_fullsized":"2023-07-06T01:19:26.630793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:44.850783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631977+0000","last_change":"2023-07-06T01:19:25.656530+0000","last_active":"2023-07-06T01:19:26.631977+0000","last_peered":"2023-07-06T01:19:26.631977+0000","last_clean":"2023-07-06T01:19:26.631977+0000","last_became_active":"2023-07-06T01:19:25.655966+0000","last_became_peered":"2023-07-06T01:19:25.655966+0000","last_unstale":"2023-07-06T01:19:26.631977+0000","last_undegraded":"2023-07-06T01:19:26.631977+0000","last_fullsized":"2023-07-06T01:19:26.631977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:10.760963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069121+0000","last_change":"2023-07-06T01:19:24.649133+0000","last_active":"2023-07-06T01:19:27.069121+0000","last_peered":"2023-07-06T01:19:27.069121+0000","last_clean":"2023-07-06T01:19:27.069121+0000","last_became_active":"2023-07-06T01:19:24.648818+0000","last_became_peered":"2023-07-06T01:19:24.648818+0000","last_unstale":"2023-07-06T01:19:27.069121+0000","last_undegraded":"2023-07-06T01:19:27.069121+0000","last_fullsized":"2023-07-06T01:19:27.069121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:45:07.932107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630689+0000","last_change":"2023-07-06T01:19:25.646210+0000","last_active":"2023-07-06T01:19:26.630689+0000","last_peered":"2023-07-06T01:19:26.630689+0000","last_clean":"2023-07-06T01:19:26.630689+0000","last_became_active":"2023-07-06T01:19:25.645912+0000","last_became_peered":"2023-07-06T01:19:25.645912+0000","last_unstale":"2023-07-06T01:19:26.630689+0000","last_undegraded":"2023-07-06T01:19:26.630689+0000","last_fullsized":"2023-07-06T01:19:26.630689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:21.566043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068646+0000","last_change":"2023-07-06T01:19:24.642544+0000","last_active":"2023-07-06T01:19:27.068646+0000","last_peered":"2023-07-06T01:19:27.068646+0000","last_clean":"2023-07-06T01:19:27.068646+0000","last_became_active":"2023-07-06T01:19:24.642267+0000","last_became_peered":"2023-07-06T01:19:24.642267+0000","last_unstale":"2023-07-06T01:19:27.068646+0000","last_undegraded":"2023-07-06T01:19:27.068646+0000","last_fullsized":"2023-07-06T01:19:27.068646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:20:59.964013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944798+0000","last_change":"2023-07-06T01:19:25.650388+0000","last_active":"2023-07-06T01:19:26.944798+0000","last_peered":"2023-07-06T01:19:26.944798+0000","last_clean":"2023-07-06T01:19:26.944798+0000","last_became_active":"2023-07-06T01:19:25.650047+0000","last_became_peered":"2023-07-06T01:19:25.650047+0000","last_unstale":"2023-07-06T01:19:26.944798+0000","last_undegraded":"2023-07-06T01:19:26.944798+0000","last_fullsized":"2023-07-06T01:19:26.944798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:08:02.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069262+0000","last_change":"2023-07-06T01:19:24.647357+0000","last_active":"2023-07-06T01:19:27.069262+0000","last_peered":"2023-07-06T01:19:27.069262+0000","last_clean":"2023-07-06T01:19:27.069262+0000","last_became_active":"2023-07-06T01:19:24.647103+0000","last_became_peered":"2023-07-06T01:19:24.647103+0000","last_unstale":"2023-07-06T01:19:27.069262+0000","last_undegraded":"2023-07-06T01:19:27.069262+0000","last_fullsized":"2023-07-06T01:19:27.069262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:35.627505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631268+0000","last_change":"2023-07-06T01:19:25.649637+0000","last_active":"2023-07-06T01:19:26.631268+0000","last_peered":"2023-07-06T01:19:26.631268+0000","last_clean":"2023-07-06T01:19:26.631268+0000","last_became_active":"2023-07-06T01:19:25.649363+0000","last_became_peered":"2023-07-06T01:19:25.649363+0000","last_unstale":"2023-07-06T01:19:26.631268+0000","last_undegraded":"2023-07-06T01:19:26.631268+0000","last_fullsized":"2023-07-06T01:19:26.631268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:44.291645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069101+0000","last_change":"2023-07-06T01:19:24.647245+0000","last_active":"2023-07-06T01:19:27.069101+0000","last_peered":"2023-07-06T01:19:27.069101+0000","last_clean":"2023-07-06T01:19:27.069101+0000","last_became_active":"2023-07-06T01:19:24.646993+0000","last_became_peered":"2023-07-06T01:19:24.646993+0000","last_unstale":"2023-07-06T01:19:27.069101+0000","last_undegraded":"2023-07-06T01:19:27.069101+0000","last_fullsized":"2023-07-06T01:19:27.069101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:17.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631382+0000","last_change":"2023-07-06T01:19:25.657736+0000","last_active":"2023-07-06T01:19:26.631382+0000","last_peered":"2023-07-06T01:19:26.631382+0000","last_clean":"2023-07-06T01:19:26.631382+0000","last_became_active":"2023-07-06T01:19:25.657511+0000","last_became_peered":"2023-07-06T01:19:25.657511+0000","last_unstale":"2023-07-06T01:19:26.631382+0000","last_undegraded":"2023-07-06T01:19:26.631382+0000","last_fullsized":"2023-07-06T01:19:26.631382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:00.621635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069462+0000","last_change":"2023-07-06T01:19:24.647904+0000","last_active":"2023-07-06T01:19:27.069462+0000","last_peered":"2023-07-06T01:19:27.069462+0000","last_clean":"2023-07-06T01:19:27.069462+0000","last_became_active":"2023-07-06T01:19:24.647621+0000","last_became_peered":"2023-07-06T01:19:24.647621+0000","last_unstale":"2023-07-06T01:19:27.069462+0000","last_undegraded":"2023-07-06T01:19:27.069462+0000","last_fullsized":"2023-07-06T01:19:27.069462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:08:01.460887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069126+0000","last_change":"2023-07-06T01:19:25.652728+0000","last_active":"2023-07-06T01:19:27.069126+0000","last_peered":"2023-07-06T01:19:27.069126+0000","last_clean":"2023-07-06T01:19:27.069126+0000","last_became_active":"2023-07-06T01:19:25.652420+0000","last_became_peered":"2023-07-06T01:19:25.652420+0000","last_unstale":"2023-07-06T01:19:27.069126+0000","last_undegraded":"2023-07-06T01:19:27.069126+0000","last_fullsized":"2023-07-06T01:19:27.069126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:40.213207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631482+0000","last_change":"2023-07-06T01:19:24.640722+0000","last_active":"2023-07-06T01:19:26.631482+0000","last_peered":"2023-07-06T01:19:26.631482+0000","last_clean":"2023-07-06T01:19:26.631482+0000","last_became_active":"2023-07-06T01:19:24.640417+0000","last_became_peered":"2023-07-06T01:19:24.640417+0000","last_unstale":"2023-07-06T01:19:26.631482+0000","last_undegraded":"2023-07-06T01:19:26.631482+0000","last_fullsized":"2023-07-06T01:19:26.631482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:24.688149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630955+0000","last_change":"2023-07-06T01:19:25.636624+0000","last_active":"2023-07-06T01:19:26.630955+0000","last_peered":"2023-07-06T01:19:26.630955+0000","last_clean":"2023-07-06T01:19:26.630955+0000","last_became_active":"2023-07-06T01:19:25.636320+0000","last_became_peered":"2023-07-06T01:19:25.636320+0000","last_unstale":"2023-07-06T01:19:26.630955+0000","last_undegraded":"2023-07-06T01:19:26.630955+0000","last_fullsized":"2023-07-06T01:19:26.630955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944677+0000","last_change":"2023-07-06T01:19:24.645728+0000","last_active":"2023-07-06T01:19:26.944677+0000","last_peered":"2023-07-06T01:19:26.944677+0000","last_clean":"2023-07-06T01:19:26.944677+0000","last_became_active":"2023-07-06T01:19:24.644479+0000","last_became_peered":"2023-07-06T01:19:24.644479+0000","last_unstale":"2023-07-06T01:19:26.944677+0000","last_undegraded":"2023-07-06T01:19:26.944677+0000","last_fullsized":"2023-07-06T01:19:26.944677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:58.274250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632444+0000","last_change":"2023-07-06T01:19:25.658759+0000","last_active":"2023-07-06T01:19:26.632444+0000","last_peered":"2023-07-06T01:19:26.632444+0000","last_clean":"2023-07-06T01:19:26.632444+0000","last_became_active":"2023-07-06T01:19:25.658482+0000","last_became_peered":"2023-07-06T01:19:25.658482+0000","last_unstale":"2023-07-06T01:19:26.632444+0000","last_undegraded":"2023-07-06T01:19:26.632444+0000","last_fullsized":"2023-07-06T01:19:26.632444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:58:41.885622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069265+0000","last_change":"2023-07-06T01:19:24.649526+0000","last_active":"2023-07-06T01:19:27.069265+0000","last_peered":"2023-07-06T01:19:27.069265+0000","last_clean":"2023-07-06T01:19:27.069265+0000","last_became_active":"2023-07-06T01:19:24.648970+0000","last_became_peered":"2023-07-06T01:19:24.648970+0000","last_unstale":"2023-07-06T01:19:27.069265+0000","last_undegraded":"2023-07-06T01:19:27.069265+0000","last_fullsized":"2023-07-06T01:19:27.069265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:41.796823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632518+0000","last_change":"2023-07-06T01:19:25.648610+0000","last_active":"2023-07-06T01:19:26.632518+0000","last_peered":"2023-07-06T01:19:26.632518+0000","last_clean":"2023-07-06T01:19:26.632518+0000","last_became_active":"2023-07-06T01:19:25.648317+0000","last_became_peered":"2023-07-06T01:19:25.648317+0000","last_unstale":"2023-07-06T01:19:26.632518+0000","last_undegraded":"2023-07-06T01:19:26.632518+0000","last_fullsized":"2023-07-06T01:19:26.632518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:47:36.660557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631774+0000","last_change":"2023-07-06T01:19:24.641218+0000","last_active":"2023-07-06T01:19:26.631774+0000","last_peered":"2023-07-06T01:19:26.631774+0000","last_clean":"2023-07-06T01:19:26.631774+0000","last_became_active":"2023-07-06T01:19:24.640780+0000","last_became_peered":"2023-07-06T01:19:24.640780+0000","last_unstale":"2023-07-06T01:19:26.631774+0000","last_undegraded":"2023-07-06T01:19:26.631774+0000","last_fullsized":"2023-07-06T01:19:26.631774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:21.142709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631764+0000","last_change":"2023-07-06T01:19:25.657935+0000","last_active":"2023-07-06T01:19:26.631764+0000","last_peered":"2023-07-06T01:19:26.631764+0000","last_clean":"2023-07-06T01:19:26.631764+0000","last_became_active":"2023-07-06T01:19:25.653472+0000","last_became_peered":"2023-07-06T01:19:25.653472+0000","last_unstale":"2023-07-06T01:19:26.631764+0000","last_undegraded":"2023-07-06T01:19:26.631764+0000","last_fullsized":"2023-07-06T01:19:26.631764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:45:08.937322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944291+0000","last_change":"2023-07-06T01:19:24.645128+0000","last_active":"2023-07-06T01:19:26.944291+0000","last_peered":"2023-07-06T01:19:26.944291+0000","last_clean":"2023-07-06T01:19:26.944291+0000","last_became_active":"2023-07-06T01:19:24.644825+0000","last_became_peered":"2023-07-06T01:19:24.644825+0000","last_unstale":"2023-07-06T01:19:26.944291+0000","last_undegraded":"2023-07-06T01:19:26.944291+0000","last_fullsized":"2023-07-06T01:19:26.944291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:14:59.616420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069000+0000","last_change":"2023-07-06T01:19:25.654566+0000","last_active":"2023-07-06T01:19:27.069000+0000","last_peered":"2023-07-06T01:19:27.069000+0000","last_clean":"2023-07-06T01:19:27.069000+0000","last_became_active":"2023-07-06T01:19:25.654238+0000","last_became_peered":"2023-07-06T01:19:25.654238+0000","last_unstale":"2023-07-06T01:19:27.069000+0000","last_undegraded":"2023-07-06T01:19:27.069000+0000","last_fullsized":"2023-07-06T01:19:27.069000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:45.855998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944614+0000","last_change":"2023-07-06T01:19:24.648921+0000","last_active":"2023-07-06T01:19:26.944614+0000","last_peered":"2023-07-06T01:19:26.944614+0000","last_clean":"2023-07-06T01:19:26.944614+0000","last_became_active":"2023-07-06T01:19:24.648676+0000","last_became_peered":"2023-07-06T01:19:24.648676+0000","last_unstale":"2023-07-06T01:19:26.944614+0000","last_undegraded":"2023-07-06T01:19:26.944614+0000","last_fullsized":"2023-07-06T01:19:26.944614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:41:33.528500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632266+0000","last_change":"2023-07-06T01:19:20.074195+0000","last_active":"2023-07-06T01:19:26.632266+0000","last_peered":"2023-07-06T01:19:26.632266+0000","last_clean":"2023-07-06T01:19:26.632266+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:19:26.632266+0000","last_undegraded":"2023-07-06T01:19:26.632266+0000","last_fullsized":"2023-07-06T01:19:26.632266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:07:56.411730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944214+0000","last_change":"2023-07-06T01:19:25.652132+0000","last_active":"2023-07-06T01:19:26.944214+0000","last_peered":"2023-07-06T01:19:26.944214+0000","last_clean":"2023-07-06T01:19:26.944214+0000","last_became_active":"2023-07-06T01:19:25.651678+0000","last_became_peered":"2023-07-06T01:19:25.651678+0000","last_unstale":"2023-07-06T01:19:26.944214+0000","last_undegraded":"2023-07-06T01:19:26.944214+0000","last_fullsized":"2023-07-06T01:19:26.944214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:45.855998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944468+0000","last_change":"2023-07-06T01:19:19.587110+0000","last_active":"2023-07-06T01:19:26.944468+0000","last_peered":"2023-07-06T01:19:26.944468+0000","last_clean":"2023-07-06T01:19:26.944468+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:19:26.944468+0000","last_undegraded":"2023-07-06T01:19:26.944468+0000","last_fullsized":"2023-07-06T01:19:26.944468+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:12.165222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.166192+0000","last_change":"2023-07-06T01:19:24.640259+0000","last_active":"2023-07-06T01:19:27.166192+0000","last_peered":"2023-07-06T01:19:27.166192+0000","last_clean":"2023-07-06T01:19:27.166192+0000","last_became_active":"2023-07-06T01:19:24.639942+0000","last_became_peered":"2023-07-06T01:19:24.639942+0000","last_unstale":"2023-07-06T01:19:27.166192+0000","last_undegraded":"2023-07-06T01:19:27.166192+0000","last_fullsized":"2023-07-06T01:19:27.166192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:30.566175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631522+0000","last_change":"2023-07-06T01:19:25.657104+0000","last_active":"2023-07-06T01:19:26.631522+0000","last_peered":"2023-07-06T01:19:26.631522+0000","last_clean":"2023-07-06T01:19:26.631522+0000","last_became_active":"2023-07-06T01:19:25.656804+0000","last_became_peered":"2023-07-06T01:19:25.656804+0000","last_unstale":"2023-07-06T01:19:26.631522+0000","last_undegraded":"2023-07-06T01:19:26.631522+0000","last_fullsized":"2023-07-06T01:19:26.631522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068648+0000","last_change":"2023-07-06T01:19:19.584242+0000","last_active":"2023-07-06T01:19:27.068648+0000","last_peered":"2023-07-06T01:19:27.068648+0000","last_clean":"2023-07-06T01:19:27.068648+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:19:27.068648+0000","last_undegraded":"2023-07-06T01:19:27.068648+0000","last_fullsized":"2023-07-06T01:19:27.068648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:27.113987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":169,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068934+0000","last_change":"2023-07-06T01:19:19.586495+0000","last_active":"2023-07-06T01:19:27.068934+0000","last_peered":"2023-07-06T01:19:27.068934+0000","last_clean":"2023-07-06T01:19:27.068934+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:19:27.068934+0000","last_undegraded":"2023-07-06T01:19:27.068934+0000","last_fullsized":"2023-07-06T01:19:27.068934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:01:53.085093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630456+0000","last_change":"2023-07-06T01:19:24.642436+0000","last_active":"2023-07-06T01:19:26.630456+0000","last_peered":"2023-07-06T01:19:26.630456+0000","last_clean":"2023-07-06T01:19:26.630456+0000","last_became_active":"2023-07-06T01:19:24.641979+0000","last_became_peered":"2023-07-06T01:19:24.641979+0000","last_unstale":"2023-07-06T01:19:26.630456+0000","last_undegraded":"2023-07-06T01:19:26.630456+0000","last_fullsized":"2023-07-06T01:19:26.630456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:43.286430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630728+0000","last_change":"2023-07-06T01:19:25.639313+0000","last_active":"2023-07-06T01:19:26.630728+0000","last_peered":"2023-07-06T01:19:26.630728+0000","last_clean":"2023-07-06T01:19:26.630728+0000","last_became_active":"2023-07-06T01:19:25.639042+0000","last_became_peered":"2023-07-06T01:19:25.639042+0000","last_unstale":"2023-07-06T01:19:26.630728+0000","last_undegraded":"2023-07-06T01:19:26.630728+0000","last_fullsized":"2023-07-06T01:19:26.630728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:40.213207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068698+0000","last_change":"2023-07-06T01:19:19.586426+0000","last_active":"2023-07-06T01:19:27.068698+0000","last_peered":"2023-07-06T01:19:27.068698+0000","last_clean":"2023-07-06T01:19:27.068698+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:19:27.068698+0000","last_undegraded":"2023-07-06T01:19:27.068698+0000","last_fullsized":"2023-07-06T01:19:27.068698+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:33.925924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068473+0000","last_change":"2023-07-06T01:19:24.643437+0000","last_active":"2023-07-06T01:19:27.068473+0000","last_peered":"2023-07-06T01:19:27.068473+0000","last_clean":"2023-07-06T01:19:27.068473+0000","last_became_active":"2023-07-06T01:19:24.642926+0000","last_became_peered":"2023-07-06T01:19:24.642926+0000","last_unstale":"2023-07-06T01:19:27.068473+0000","last_undegraded":"2023-07-06T01:19:27.068473+0000","last_fullsized":"2023-07-06T01:19:27.068473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:17.543019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068999+0000","last_change":"2023-07-06T01:19:25.656081+0000","last_active":"2023-07-06T01:19:27.068999+0000","last_peered":"2023-07-06T01:19:27.068999+0000","last_clean":"2023-07-06T01:19:27.068999+0000","last_became_active":"2023-07-06T01:19:25.655801+0000","last_became_peered":"2023-07-06T01:19:25.655801+0000","last_unstale":"2023-07-06T01:19:27.068999+0000","last_undegraded":"2023-07-06T01:19:27.068999+0000","last_fullsized":"2023-07-06T01:19:27.068999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:10.760963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068626+0000","last_change":"2023-07-06T01:19:19.586626+0000","last_active":"2023-07-06T01:19:27.068626+0000","last_peered":"2023-07-06T01:19:27.068626+0000","last_clean":"2023-07-06T01:19:27.068626+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:19:27.068626+0000","last_undegraded":"2023-07-06T01:19:27.068626+0000","last_fullsized":"2023-07-06T01:19:27.068626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:30.708237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944310+0000","last_change":"2023-07-06T01:19:24.644389+0000","last_active":"2023-07-06T01:19:26.944310+0000","last_peered":"2023-07-06T01:19:26.944310+0000","last_clean":"2023-07-06T01:19:26.944310+0000","last_became_active":"2023-07-06T01:19:24.644113+0000","last_became_peered":"2023-07-06T01:19:24.644113+0000","last_unstale":"2023-07-06T01:19:26.944310+0000","last_undegraded":"2023-07-06T01:19:26.944310+0000","last_fullsized":"2023-07-06T01:19:26.944310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:30.566175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069023+0000","last_change":"2023-07-06T01:19:25.654127+0000","last_active":"2023-07-06T01:19:27.069023+0000","last_peered":"2023-07-06T01:19:27.069023+0000","last_clean":"2023-07-06T01:19:27.069023+0000","last_became_active":"2023-07-06T01:19:25.653903+0000","last_became_peered":"2023-07-06T01:19:25.653903+0000","last_unstale":"2023-07-06T01:19:27.069023+0000","last_undegraded":"2023-07-06T01:19:27.069023+0000","last_fullsized":"2023-07-06T01:19:27.069023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:25.693365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631434+0000","last_change":"2023-07-06T01:19:19.584086+0000","last_active":"2023-07-06T01:19:26.631434+0000","last_peered":"2023-07-06T01:19:26.631434+0000","last_clean":"2023-07-06T01:19:26.631434+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:19:26.631434+0000","last_undegraded":"2023-07-06T01:19:26.631434+0000","last_fullsized":"2023-07-06T01:19:26.631434+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:36.747665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632572+0000","last_change":"2023-07-06T01:19:24.648405+0000","last_active":"2023-07-06T01:19:26.632572+0000","last_peered":"2023-07-06T01:19:26.632572+0000","last_clean":"2023-07-06T01:19:26.632572+0000","last_became_active":"2023-07-06T01:19:24.648161+0000","last_became_peered":"2023-07-06T01:19:24.648161+0000","last_unstale":"2023-07-06T01:19:26.632572+0000","last_undegraded":"2023-07-06T01:19:26.632572+0000","last_fullsized":"2023-07-06T01:19:26.632572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:39:24.607769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069089+0000","last_change":"2023-07-06T01:19:25.641885+0000","last_active":"2023-07-06T01:19:27.069089+0000","last_peered":"2023-07-06T01:19:27.069089+0000","last_clean":"2023-07-06T01:19:27.069089+0000","last_became_active":"2023-07-06T01:19:25.641570+0000","last_became_peered":"2023-07-06T01:19:25.641570+0000","last_unstale":"2023-07-06T01:19:27.069089+0000","last_undegraded":"2023-07-06T01:19:27.069089+0000","last_fullsized":"2023-07-06T01:19:27.069089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:09.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069072+0000","last_change":"2023-07-06T01:19:19.586424+0000","last_active":"2023-07-06T01:19:27.069072+0000","last_peered":"2023-07-06T01:19:27.069072+0000","last_clean":"2023-07-06T01:19:27.069072+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:19:27.069072+0000","last_undegraded":"2023-07-06T01:19:27.069072+0000","last_fullsized":"2023-07-06T01:19:27.069072+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:15.511670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068948+0000","last_change":"2023-07-06T01:19:24.644635+0000","last_active":"2023-07-06T01:19:27.068948+0000","last_peered":"2023-07-06T01:19:27.068948+0000","last_clean":"2023-07-06T01:19:27.068948+0000","last_became_active":"2023-07-06T01:19:24.644344+0000","last_became_peered":"2023-07-06T01:19:24.644344+0000","last_unstale":"2023-07-06T01:19:27.068948+0000","last_undegraded":"2023-07-06T01:19:27.068948+0000","last_fullsized":"2023-07-06T01:19:27.068948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:39.207991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631123+0000","last_change":"2023-07-06T01:19:25.657020+0000","last_active":"2023-07-06T01:19:26.631123+0000","last_peered":"2023-07-06T01:19:26.631123+0000","last_clean":"2023-07-06T01:19:26.631123+0000","last_became_active":"2023-07-06T01:19:25.656084+0000","last_became_peered":"2023-07-06T01:19:25.656084+0000","last_unstale":"2023-07-06T01:19:26.631123+0000","last_undegraded":"2023-07-06T01:19:26.631123+0000","last_fullsized":"2023-07-06T01:19:26.631123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:22.147924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069477+0000","last_change":"2023-07-06T01:19:19.583699+0000","last_active":"2023-07-06T01:19:27.069477+0000","last_peered":"2023-07-06T01:19:27.069477+0000","last_clean":"2023-07-06T01:19:27.069477+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:19:27.069477+0000","last_undegraded":"2023-07-06T01:19:27.069477+0000","last_fullsized":"2023-07-06T01:19:27.069477+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:04.706590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631992+0000","last_change":"2023-07-06T01:19:24.642998+0000","last_active":"2023-07-06T01:19:26.631992+0000","last_peered":"2023-07-06T01:19:26.631992+0000","last_clean":"2023-07-06T01:19:26.631992+0000","last_became_active":"2023-07-06T01:19:24.642674+0000","last_became_peered":"2023-07-06T01:19:24.642674+0000","last_unstale":"2023-07-06T01:19:26.631992+0000","last_undegraded":"2023-07-06T01:19:26.631992+0000","last_fullsized":"2023-07-06T01:19:26.631992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:44.850783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068822+0000","last_change":"2023-07-06T01:19:25.651577+0000","last_active":"2023-07-06T01:19:27.068822+0000","last_peered":"2023-07-06T01:19:27.068822+0000","last_clean":"2023-07-06T01:19:27.068822+0000","last_became_active":"2023-07-06T01:19:25.651299+0000","last_became_peered":"2023-07-06T01:19:25.651299+0000","last_unstale":"2023-07-06T01:19:27.068822+0000","last_undegraded":"2023-07-06T01:19:27.068822+0000","last_fullsized":"2023-07-06T01:19:27.068822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:25.693365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631194+0000","last_change":"2023-07-06T01:19:24.648687+0000","last_active":"2023-07-06T01:19:26.631194+0000","last_peered":"2023-07-06T01:19:26.631194+0000","last_clean":"2023-07-06T01:19:26.631194+0000","last_became_active":"2023-07-06T01:19:24.648363+0000","last_became_peered":"2023-07-06T01:19:24.648363+0000","last_unstale":"2023-07-06T01:19:26.631194+0000","last_undegraded":"2023-07-06T01:19:26.631194+0000","last_fullsized":"2023-07-06T01:19:26.631194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:24.719467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069057+0000","last_change":"2023-07-06T01:19:25.646102+0000","last_active":"2023-07-06T01:19:27.069057+0000","last_peered":"2023-07-06T01:19:27.069057+0000","last_clean":"2023-07-06T01:19:27.069057+0000","last_became_active":"2023-07-06T01:19:25.645911+0000","last_became_peered":"2023-07-06T01:19:25.645911+0000","last_unstale":"2023-07-06T01:19:27.069057+0000","last_undegraded":"2023-07-06T01:19:27.069057+0000","last_fullsized":"2023-07-06T01:19:27.069057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:58:05.681285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631273+0000","last_change":"2023-07-06T01:19:24.645915+0000","last_active":"2023-07-06T01:19:26.631273+0000","last_peered":"2023-07-06T01:19:26.631273+0000","last_clean":"2023-07-06T01:19:26.631273+0000","last_became_active":"2023-07-06T01:19:24.645626+0000","last_became_peered":"2023-07-06T01:19:24.645626+0000","last_unstale":"2023-07-06T01:19:26.631273+0000","last_undegraded":"2023-07-06T01:19:26.631273+0000","last_fullsized":"2023-07-06T01:19:26.631273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:32.163145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632582+0000","last_change":"2023-07-06T01:19:25.658031+0000","last_active":"2023-07-06T01:19:26.632582+0000","last_peered":"2023-07-06T01:19:26.632582+0000","last_clean":"2023-07-06T01:19:26.632582+0000","last_became_active":"2023-07-06T01:19:25.657733+0000","last_became_peered":"2023-07-06T01:19:25.657733+0000","last_unstale":"2023-07-06T01:19:26.632582+0000","last_undegraded":"2023-07-06T01:19:26.632582+0000","last_fullsized":"2023-07-06T01:19:26.632582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:35:30.587396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632124+0000","last_change":"2023-07-06T01:19:24.645784+0000","last_active":"2023-07-06T01:19:26.632124+0000","last_peered":"2023-07-06T01:19:26.632124+0000","last_clean":"2023-07-06T01:19:26.632124+0000","last_became_active":"2023-07-06T01:19:24.645447+0000","last_became_peered":"2023-07-06T01:19:24.645447+0000","last_unstale":"2023-07-06T01:19:26.632124+0000","last_undegraded":"2023-07-06T01:19:26.632124+0000","last_fullsized":"2023-07-06T01:19:26.632124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:08.965590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632116+0000","last_change":"2023-07-06T01:19:25.653462+0000","last_active":"2023-07-06T01:19:26.632116+0000","last_peered":"2023-07-06T01:19:26.632116+0000","last_clean":"2023-07-06T01:19:26.632116+0000","last_became_active":"2023-07-06T01:19:25.652772+0000","last_became_peered":"2023-07-06T01:19:25.652772+0000","last_unstale":"2023-07-06T01:19:26.632116+0000","last_undegraded":"2023-07-06T01:19:26.632116+0000","last_fullsized":"2023-07-06T01:19:26.632116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:01:59.139466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944733+0000","last_change":"2023-07-06T01:19:24.642392+0000","last_active":"2023-07-06T01:19:26.944733+0000","last_peered":"2023-07-06T01:19:26.944733+0000","last_clean":"2023-07-06T01:19:26.944733+0000","last_became_active":"2023-07-06T01:19:24.642122+0000","last_became_peered":"2023-07-06T01:19:24.642122+0000","last_unstale":"2023-07-06T01:19:26.944733+0000","last_undegraded":"2023-07-06T01:19:26.944733+0000","last_fullsized":"2023-07-06T01:19:26.944733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:01:58.134250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944450+0000","last_change":"2023-07-06T01:19:25.650007+0000","last_active":"2023-07-06T01:19:26.944450+0000","last_peered":"2023-07-06T01:19:26.944450+0000","last_clean":"2023-07-06T01:19:26.944450+0000","last_became_active":"2023-07-06T01:19:25.649108+0000","last_became_peered":"2023-07-06T01:19:25.649108+0000","last_unstale":"2023-07-06T01:19:26.944450+0000","last_undegraded":"2023-07-06T01:19:26.944450+0000","last_fullsized":"2023-07-06T01:19:26.944450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:22.147924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630751+0000","last_change":"2023-07-06T01:19:24.642694+0000","last_active":"2023-07-06T01:19:26.630751+0000","last_peered":"2023-07-06T01:19:26.630751+0000","last_clean":"2023-07-06T01:19:26.630751+0000","last_became_active":"2023-07-06T01:19:24.642130+0000","last_became_peered":"2023-07-06T01:19:24.642130+0000","last_unstale":"2023-07-06T01:19:26.630751+0000","last_undegraded":"2023-07-06T01:19:26.630751+0000","last_fullsized":"2023-07-06T01:19:26.630751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:09.755748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631214+0000","last_change":"2023-07-06T01:19:25.653965+0000","last_active":"2023-07-06T01:19:26.631214+0000","last_peered":"2023-07-06T01:19:26.631214+0000","last_clean":"2023-07-06T01:19:26.631214+0000","last_became_active":"2023-07-06T01:19:25.653448+0000","last_became_peered":"2023-07-06T01:19:25.653448+0000","last_unstale":"2023-07-06T01:19:26.631214+0000","last_undegraded":"2023-07-06T01:19:26.631214+0000","last_fullsized":"2023-07-06T01:19:26.631214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:09.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068713+0000","last_change":"2023-07-06T01:19:25.652413+0000","last_active":"2023-07-06T01:19:27.068713+0000","last_peered":"2023-07-06T01:19:27.068713+0000","last_clean":"2023-07-06T01:19:27.068713+0000","last_became_active":"2023-07-06T01:19:25.652051+0000","last_became_peered":"2023-07-06T01:19:25.652051+0000","last_unstale":"2023-07-06T01:19:27.068713+0000","last_undegraded":"2023-07-06T01:19:27.068713+0000","last_fullsized":"2023-07-06T01:19:27.068713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:44.291645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944695+0000","last_change":"2023-07-06T01:19:25.652994+0000","last_active":"2023-07-06T01:19:26.944695+0000","last_peered":"2023-07-06T01:19:26.944695+0000","last_clean":"2023-07-06T01:19:26.944695+0000","last_became_active":"2023-07-06T01:19:25.652484+0000","last_became_peered":"2023-07-06T01:19:25.652484+0000","last_unstale":"2023-07-06T01:19:26.944695+0000","last_undegraded":"2023-07-06T01:19:26.944695+0000","last_fullsized":"2023-07-06T01:19:26.944695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:14.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631822+0000","last_change":"2023-07-06T01:19:25.656779+0000","last_active":"2023-07-06T01:19:26.631822+0000","last_peered":"2023-07-06T01:19:26.631822+0000","last_clean":"2023-07-06T01:19:26.631822+0000","last_became_active":"2023-07-06T01:19:25.653104+0000","last_became_peered":"2023-07-06T01:19:25.653104+0000","last_unstale":"2023-07-06T01:19:26.631822+0000","last_undegraded":"2023-07-06T01:19:26.631822+0000","last_fullsized":"2023-07-06T01:19:26.631822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:41:34.533716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944583+0000","last_change":"2023-07-06T01:19:25.653857+0000","last_active":"2023-07-06T01:19:26.944583+0000","last_peered":"2023-07-06T01:19:26.944583+0000","last_clean":"2023-07-06T01:19:26.944583+0000","last_became_active":"2023-07-06T01:19:25.652976+0000","last_became_peered":"2023-07-06T01:19:25.652976+0000","last_unstale":"2023-07-06T01:19:26.944583+0000","last_undegraded":"2023-07-06T01:19:26.944583+0000","last_fullsized":"2023-07-06T01:19:26.944583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:25.724682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068930+0000","last_change":"2023-07-06T01:19:25.647146+0000","last_active":"2023-07-06T01:19:27.068930+0000","last_peered":"2023-07-06T01:19:27.068930+0000","last_clean":"2023-07-06T01:19:27.068930+0000","last_became_active":"2023-07-06T01:19:25.646863+0000","last_became_peered":"2023-07-06T01:19:25.646863+0000","last_unstale":"2023-07-06T01:19:27.068930+0000","last_undegraded":"2023-07-06T01:19:27.068930+0000","last_fullsized":"2023-07-06T01:19:27.068930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:21.566043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":670687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":233472,"data_stored":80159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":806912,"data_stored":668907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5180000000000007}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.963000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.397}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.460000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9890000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.097}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1429999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.2430000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":78539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":78629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":79191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":78539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:19:34.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:19:34.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.733+0000 7f9d50d39700 1 Processor -- start 2023-07-06T01:19:34.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.733+0000 7f9d50d39700 1 -- start start 2023-07-06T01:19:34.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.734+0000 7f9d50d39700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d4c0af2a0 0x7f9d4c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:34.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.734+0000 7f9d50d39700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9d4c06a000 0x7f9d4c06a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:34.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.734+0000 7f9d50d39700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4c0af670 con 0x7f9d4c0cbc10 2023-07-06T01:19:34.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.734+0000 7f9d50d39700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d4c06a9e0 con 0x7f9d4c06a000 2023-07-06T01:19:34.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.734+0000 7f9d50d39700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c0cc7a0 con 0x7f9d4c0af2a0 2023-07-06T01:19:34.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d4ad9d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9d4c0cbc10 0x7f9d4c0cc090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36524/0 (socket says 172.21.15.22:36524) 2023-07-06T01:19:34.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d4ad9d700 1 -- 172.21.15.22:0/910243294 learned_addr learned my addr 172.21.15.22:0/910243294 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:34.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d49d9b700 1 --2- 172.21.15.22:0/910243294 >> v2:172.21.15.103:3300/0 conn(0x7f9d4c06a000 0x7f9d4c06a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:34.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d4a59c700 1 --2- 172.21.15.22:0/910243294 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d4c0af2a0 0x7f9d4c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:34.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d4959a700 1 -- 172.21.15.22:0/910243294 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733360566 0 0) 0x7f9d4c0af670 con 0x7f9d4c0cbc10 2023-07-06T01:19:34.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d4959a700 1 -- 172.21.15.22:0/910243294 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d380035b0 con 0x7f9d4c0cbc10 2023-07-06T01:19:34.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d49d9b700 1 -- 172.21.15.22:0/910243294 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d4c0af2a0 msgr2=0x7f9d4c06caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:34.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d49d9b700 1 --2- 172.21.15.22:0/910243294 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d4c0af2a0 0x7f9d4c06caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d49d9b700 1 -- 172.21.15.22:0/910243294 >> v1:172.21.15.22:6789/0 conn(0x7f9d4c0cbc10 legacy=0x7f9d4c0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d4959a700 1 -- 172.21.15.22:0/910243294 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 111845177 0 0) 0x7f9d380035b0 con 0x7f9d4c0cbc10 2023-07-06T01:19:34.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d49d9b700 1 -- 172.21.15.22:0/910243294 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c06d030 con 0x7f9d4c06a000 2023-07-06T01:19:34.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d4a59c700 1 --2- 172.21.15.22:0/910243294 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d4c0af2a0 0x7f9d4c06caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:34.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d49d9b700 1 --2- 172.21.15.22:0/910243294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d4c06a000 0x7f9d4c06a4a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009840 tx=0x7f9d34009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b766391252c3c029 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:34.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.735+0000 7f9d4959a700 1 -- 172.21.15.22:0/910243294 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d440019f0 con 0x7f9d4c06a000 2023-07-06T01:19:34.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.736+0000 7f9d4959a700 1 -- 172.21.15.22:0/910243294 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d34004030 con 0x7f9d4c06a000 2023-07-06T01:19:34.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.736+0000 7f9d4959a700 1 -- 172.21.15.22:0/910243294 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d340043c0 con 0x7f9d4c06a000 2023-07-06T01:19:34.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.736+0000 7f9d50d39700 1 -- 172.21.15.22:0/910243294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d4c06a000 msgr2=0x7f9d4c06a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.736+0000 7f9d50d39700 1 --2- 172.21.15.22:0/910243294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d4c06a000 0x7f9d4c06a4a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009840 tx=0x7f9d34009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:34.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.736+0000 7f9d50d39700 1 -- 172.21.15.22:0/910243294 shutdown_connections 2023-07-06T01:19:34.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.736+0000 7f9d50d39700 1 --2- 172.21.15.22:0/910243294 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d4c0af2a0 0x7f9d4c06caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.736+0000 7f9d50d39700 1 --2- 172.21.15.22:0/910243294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d4c06a000 0x7f9d4c06a4a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.736+0000 7f9d50d39700 1 -- 172.21.15.22:0/910243294 >> 172.21.15.22:0/910243294 conn(0x7f9d4c0b1890 msgr2=0x7f9d4c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:34.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.736+0000 7f9d50d39700 1 -- 172.21.15.22:0/910243294 shutdown_connections 2023-07-06T01:19:34.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.736+0000 7f9d50d39700 1 -- 172.21.15.22:0/910243294 wait complete. 2023-07-06T01:19:34.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.737+0000 7f9d50d39700 1 Processor -- start 2023-07-06T01:19:34.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.738+0000 7f9d50d39700 1 -- start start 2023-07-06T01:19:34.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.738+0000 7f9d50d39700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c06a000 0x7f9d4c1f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:34.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.738+0000 7f9d50d39700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d4c0af2a0 0x7f9d4c15a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:34.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.738+0000 7f9d50d39700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d4c0cbc10 0x7f9d4c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:34.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.738+0000 7f9d50d39700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c15aa40 con 0x7f9d4c06a000 2023-07-06T01:19:34.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.738+0000 7f9d50d39700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c1f3110 con 0x7f9d4c0af2a0 2023-07-06T01:19:34.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.738+0000 7f9d50d39700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c1f5b80 con 0x7f9d4c0cbc10 2023-07-06T01:19:34.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.738+0000 7f9d4a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c06a000 0x7f9d4c1f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:34.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.738+0000 7f9d4a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c06a000 0x7f9d4c1f6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45788/0 (socket says 172.21.15.22:45788) 2023-07-06T01:19:34.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.738+0000 7f9d4a59c700 1 -- 172.21.15.22:0/1895551904 learned_addr learned my addr 172.21.15.22:0/1895551904 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:34.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.739+0000 7f9d49d9b700 1 --2- 172.21.15.22:0/1895551904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d4c0af2a0 0x7f9d4c15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:34.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.739+0000 7f9d4a59c700 1 -- 172.21.15.22:0/1895551904 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d4c0cbc10 msgr2=0x7f9d4c0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.739+0000 7f9d4a59c700 1 --2- 172.21.15.22:0/1895551904 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d4c0cbc10 0x7f9d4c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.739+0000 7f9d4a59c700 1 -- 172.21.15.22:0/1895551904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d4c0af2a0 msgr2=0x7f9d4c15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.739+0000 7f9d4a59c700 1 --2- 172.21.15.22:0/1895551904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d4c0af2a0 0x7f9d4c15a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.739+0000 7f9d4a59c700 1 -- 172.21.15.22:0/1895551904 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c0094a0 con 0x7f9d4c06a000 2023-07-06T01:19:34.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.739+0000 7f9d4a59c700 1 --2- 172.21.15.22:0/1895551904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c06a000 0x7f9d4c1f6660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00ebe0 tx=0x7f9d3c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:34.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.739+0000 7f9d42ffd700 1 -- 172.21.15.22:0/1895551904 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00c910 con 0x7f9d4c06a000 2023-07-06T01:19:34.749 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.739+0000 7f9d42ffd700 1 -- 172.21.15.22:0/1895551904 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c00ca70 con 0x7f9d4c06a000 2023-07-06T01:19:34.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.739+0000 7f9d42ffd700 1 -- 172.21.15.22:0/1895551904 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00cde0 con 0x7f9d4c06a000 2023-07-06T01:19:34.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.739+0000 7f9d50d39700 1 -- 172.21.15.22:0/1895551904 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d340094a0 con 0x7f9d4c06a000 2023-07-06T01:19:34.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.739+0000 7f9d50d39700 1 -- 172.21.15.22:0/1895551904 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4c1f6140 con 0x7f9d4c06a000 2023-07-06T01:19:34.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.740+0000 7f9d50d39700 1 -- 172.21.15.22:0/1895551904 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4c071110 con 0x7f9d4c06a000 2023-07-06T01:19:34.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.740+0000 7f9d42ffd700 1 -- 172.21.15.22:0/1895551904 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d3c013470 con 0x7f9d4c06a000 2023-07-06T01:19:34.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.741+0000 7f9d42ffd700 1 --2- 172.21.15.22:0/1895551904 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9d2c06ace0 0x7f9d2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:34.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.741+0000 7f9d42ffd700 1 -- 172.21.15.22:0/1895551904 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9d3c0895c0 con 0x7f9d4c06a000 2023-07-06T01:19:34.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.741+0000 7f9d49d9b700 1 --2- 172.21.15.22:0/1895551904 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9d2c06ace0 0x7f9d2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:34.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.741+0000 7f9d49d9b700 1 --2- 172.21.15.22:0/1895551904 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9d2c06ace0 0x7f9d2c06d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9d34000c00 tx=0x7f9d34008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:34.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.745+0000 7f9d42ffd700 1 -- 172.21.15.22:0/1895551904 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9d3c054700 con 0x7f9d4c06a000 2023-07-06T01:19:34.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.938+0000 7f9d50d39700 1 -- 172.21.15.22:0/1895551904 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9d4c0cd560 con 0x7f9d2c06ace0 2023-07-06T01:19:34.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.946+0000 7f9d42ffd700 1 -- 172.21.15.22:0/1895551904 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286822 (secure 0 0 0) 0x7f9d340041a0 con 0x7f9d2c06ace0 2023-07-06T01:19:34.953 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:34.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.949+0000 7f9d50d39700 1 -- 172.21.15.22:0/1895551904 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9d2c06ace0 msgr2=0x7f9d2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.949+0000 7f9d50d39700 1 --2- 172.21.15.22:0/1895551904 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9d2c06ace0 0x7f9d2c06d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9d34000c00 tx=0x7f9d34008030 comp rx=0 tx=0).stop 2023-07-06T01:19:34.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.949+0000 7f9d50d39700 1 -- 172.21.15.22:0/1895551904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c06a000 msgr2=0x7f9d4c1f6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:34.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.949+0000 7f9d50d39700 1 --2- 172.21.15.22:0/1895551904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c06a000 0x7f9d4c1f6660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00ebe0 tx=0x7f9d3c009ec0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.949+0000 7f9d50d39700 1 -- 172.21.15.22:0/1895551904 shutdown_connections 2023-07-06T01:19:34.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.949+0000 7f9d50d39700 1 --2- 172.21.15.22:0/1895551904 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d4c0cbc10 0x7f9d4c0b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.949+0000 7f9d50d39700 1 --2- 172.21.15.22:0/1895551904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d4c06a000 0x7f9d4c1f6660 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.949+0000 7f9d50d39700 1 --2- 172.21.15.22:0/1895551904 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9d2c06ace0 0x7f9d2c06d190 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.949+0000 7f9d50d39700 1 --2- 172.21.15.22:0/1895551904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d4c0af2a0 0x7f9d4c15a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:34.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.949+0000 7f9d50d39700 1 -- 172.21.15.22:0/1895551904 >> 172.21.15.22:0/1895551904 conn(0x7f9d4c0b1890 msgr2=0x7f9d4c0cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:34.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.949+0000 7f9d50d39700 1 -- 172.21.15.22:0/1895551904 shutdown_connections 2023-07-06T01:19:34.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:34.949+0000 7f9d50d39700 1 -- 172.21.15.22:0/1895551904 wait complete. 2023-07-06T01:19:34.957 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:19:34.964 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-07-06T01:19:33.999628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218504,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756216,"statfs":{"total":773094113280,"available":772870365184,"internally_reserved":0,"allocated":2957312,"data_stored":1813772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.369917"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068502+0000","last_change":"2023-07-06T01:19:25.653405+0000","last_active":"2023-07-06T01:19:27.068502+0000","last_peered":"2023-07-06T01:19:27.068502+0000","last_clean":"2023-07-06T01:19:27.068502+0000","last_became_active":"2023-07-06T01:19:25.653093+0000","last_became_peered":"2023-07-06T01:19:25.653093+0000","last_unstale":"2023-07-06T01:19:27.068502+0000","last_undegraded":"2023-07-06T01:19:27.068502+0000","last_fullsized":"2023-07-06T01:19:27.068502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:17.116328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632301+0000","last_change":"2023-07-06T01:19:25.649747+0000","last_active":"2023-07-06T01:19:26.632301+0000","last_peered":"2023-07-06T01:19:26.632301+0000","last_clean":"2023-07-06T01:19:26.632301+0000","last_became_active":"2023-07-06T01:19:25.649468+0000","last_became_peered":"2023-07-06T01:19:25.649468+0000","last_unstale":"2023-07-06T01:19:26.632301+0000","last_undegraded":"2023-07-06T01:19:26.632301+0000","last_fullsized":"2023-07-06T01:19:26.632301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:00.621635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632173+0000","last_change":"2023-07-06T01:19:25.660551+0000","last_active":"2023-07-06T01:19:26.632173+0000","last_peered":"2023-07-06T01:19:26.632173+0000","last_clean":"2023-07-06T01:19:26.632173+0000","last_became_active":"2023-07-06T01:19:25.660278+0000","last_became_peered":"2023-07-06T01:19:25.660278+0000","last_unstale":"2023-07-06T01:19:26.632173+0000","last_undegraded":"2023-07-06T01:19:26.632173+0000","last_fullsized":"2023-07-06T01:19:26.632173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:36.632721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068877+0000","last_change":"2023-07-06T01:19:25.647264+0000","last_active":"2023-07-06T01:19:27.068877+0000","last_peered":"2023-07-06T01:19:27.068877+0000","last_clean":"2023-07-06T01:19:27.068877+0000","last_became_active":"2023-07-06T01:19:25.646962+0000","last_became_peered":"2023-07-06T01:19:25.646962+0000","last_unstale":"2023-07-06T01:19:27.068877+0000","last_undegraded":"2023-07-06T01:19:27.068877+0000","last_fullsized":"2023-07-06T01:19:27.068877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:33.168360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631056+0000","last_change":"2023-07-06T01:19:25.655455+0000","last_active":"2023-07-06T01:19:26.631056+0000","last_peered":"2023-07-06T01:19:26.631056+0000","last_clean":"2023-07-06T01:19:26.631056+0000","last_became_active":"2023-07-06T01:19:25.655137+0000","last_became_peered":"2023-07-06T01:19:25.655137+0000","last_unstale":"2023-07-06T01:19:26.631056+0000","last_undegraded":"2023-07-06T01:19:26.631056+0000","last_fullsized":"2023-07-06T01:19:26.631056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:39.980297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631681+0000","last_change":"2023-07-06T01:19:25.660076+0000","last_active":"2023-07-06T01:19:26.631681+0000","last_peered":"2023-07-06T01:19:26.631681+0000","last_clean":"2023-07-06T01:19:26.631681+0000","last_became_active":"2023-07-06T01:19:25.659798+0000","last_became_peered":"2023-07-06T01:19:25.659798+0000","last_unstale":"2023-07-06T01:19:26.631681+0000","last_undegraded":"2023-07-06T01:19:26.631681+0000","last_fullsized":"2023-07-06T01:19:26.631681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:25.724682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068900+0000","last_change":"2023-07-06T01:19:25.649766+0000","last_active":"2023-07-06T01:19:27.068900+0000","last_peered":"2023-07-06T01:19:27.068900+0000","last_clean":"2023-07-06T01:19:27.068900+0000","last_became_active":"2023-07-06T01:19:25.649497+0000","last_became_peered":"2023-07-06T01:19:25.649497+0000","last_unstale":"2023-07-06T01:19:27.068900+0000","last_undegraded":"2023-07-06T01:19:27.068900+0000","last_fullsized":"2023-07-06T01:19:27.068900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:25.724682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631105+0000","last_change":"2023-07-06T01:19:25.657379+0000","last_active":"2023-07-06T01:19:26.631105+0000","last_peered":"2023-07-06T01:19:26.631105+0000","last_clean":"2023-07-06T01:19:26.631105+0000","last_became_active":"2023-07-06T01:19:25.657044+0000","last_became_peered":"2023-07-06T01:19:25.657044+0000","last_unstale":"2023-07-06T01:19:26.631105+0000","last_undegraded":"2023-07-06T01:19:26.631105+0000","last_fullsized":"2023-07-06T01:19:26.631105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:21:00.969228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630543+0000","last_change":"2023-07-06T01:19:25.654446+0000","last_active":"2023-07-06T01:19:26.630543+0000","last_peered":"2023-07-06T01:19:26.630543+0000","last_clean":"2023-07-06T01:19:26.630543+0000","last_became_active":"2023-07-06T01:19:25.653834+0000","last_became_peered":"2023-07-06T01:19:25.653834+0000","last_unstale":"2023-07-06T01:19:26.630543+0000","last_undegraded":"2023-07-06T01:19:26.630543+0000","last_fullsized":"2023-07-06T01:19:26.630543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:33.168360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632483+0000","last_change":"2023-07-06T01:19:25.648054+0000","last_active":"2023-07-06T01:19:26.632483+0000","last_peered":"2023-07-06T01:19:26.632483+0000","last_clean":"2023-07-06T01:19:26.632483+0000","last_became_active":"2023-07-06T01:19:25.647795+0000","last_became_peered":"2023-07-06T01:19:25.647795+0000","last_unstale":"2023-07-06T01:19:26.632483+0000","last_undegraded":"2023-07-06T01:19:26.632483+0000","last_fullsized":"2023-07-06T01:19:26.632483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632291+0000","last_change":"2023-07-06T01:19:25.657523+0000","last_active":"2023-07-06T01:19:26.632291+0000","last_peered":"2023-07-06T01:19:26.632291+0000","last_clean":"2023-07-06T01:19:26.632291+0000","last_became_active":"2023-07-06T01:19:25.657258+0000","last_became_peered":"2023-07-06T01:19:25.657258+0000","last_unstale":"2023-07-06T01:19:26.632291+0000","last_undegraded":"2023-07-06T01:19:26.632291+0000","last_fullsized":"2023-07-06T01:19:26.632291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:59.279465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068838+0000","last_change":"2023-07-06T01:19:25.660991+0000","last_active":"2023-07-06T01:19:27.068838+0000","last_peered":"2023-07-06T01:19:27.068838+0000","last_clean":"2023-07-06T01:19:27.068838+0000","last_became_active":"2023-07-06T01:19:25.660654+0000","last_became_peered":"2023-07-06T01:19:25.660654+0000","last_unstale":"2023-07-06T01:19:27.068838+0000","last_undegraded":"2023-07-06T01:19:27.068838+0000","last_fullsized":"2023-07-06T01:19:27.068838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:43:26.720400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068325+0000","last_change":"2023-07-06T01:19:25.654120+0000","last_active":"2023-07-06T01:19:27.068325+0000","last_peered":"2023-07-06T01:19:27.068325+0000","last_clean":"2023-07-06T01:19:27.068325+0000","last_became_active":"2023-07-06T01:19:25.653844+0000","last_became_peered":"2023-07-06T01:19:25.653844+0000","last_unstale":"2023-07-06T01:19:27.068325+0000","last_undegraded":"2023-07-06T01:19:27.068325+0000","last_fullsized":"2023-07-06T01:19:27.068325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:42:37.007207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631522+0000","last_change":"2023-07-06T01:19:25.650622+0000","last_active":"2023-07-06T01:19:26.631522+0000","last_peered":"2023-07-06T01:19:26.631522+0000","last_clean":"2023-07-06T01:19:26.631522+0000","last_became_active":"2023-07-06T01:19:25.650310+0000","last_became_peered":"2023-07-06T01:19:25.650310+0000","last_unstale":"2023-07-06T01:19:26.631522+0000","last_undegraded":"2023-07-06T01:19:26.631522+0000","last_fullsized":"2023-07-06T01:19:26.631522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:21.566043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069099+0000","last_change":"2023-07-06T01:19:25.651765+0000","last_active":"2023-07-06T01:19:27.069099+0000","last_peered":"2023-07-06T01:19:27.069099+0000","last_clean":"2023-07-06T01:19:27.069099+0000","last_became_active":"2023-07-06T01:19:25.651244+0000","last_became_peered":"2023-07-06T01:19:25.651244+0000","last_unstale":"2023-07-06T01:19:27.069099+0000","last_undegraded":"2023-07-06T01:19:27.069099+0000","last_fullsized":"2023-07-06T01:19:27.069099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:41:34.533716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631383+0000","last_change":"2023-07-06T01:19:25.663299+0000","last_active":"2023-07-06T01:19:26.631383+0000","last_peered":"2023-07-06T01:19:26.631383+0000","last_clean":"2023-07-06T01:19:26.631383+0000","last_became_active":"2023-07-06T01:19:25.663032+0000","last_became_peered":"2023-07-06T01:19:25.663032+0000","last_unstale":"2023-07-06T01:19:26.631383+0000","last_undegraded":"2023-07-06T01:19:26.631383+0000","last_fullsized":"2023-07-06T01:19:26.631383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:31.571390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630591+0000","last_change":"2023-07-06T01:19:25.654669+0000","last_active":"2023-07-06T01:19:26.630591+0000","last_peered":"2023-07-06T01:19:26.630591+0000","last_clean":"2023-07-06T01:19:26.630591+0000","last_became_active":"2023-07-06T01:19:25.654161+0000","last_became_peered":"2023-07-06T01:19:25.654161+0000","last_unstale":"2023-07-06T01:19:26.630591+0000","last_undegraded":"2023-07-06T01:19:26.630591+0000","last_fullsized":"2023-07-06T01:19:26.630591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:42.802038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631594+0000","last_change":"2023-07-06T01:19:25.651140+0000","last_active":"2023-07-06T01:19:26.631594+0000","last_peered":"2023-07-06T01:19:26.631594+0000","last_clean":"2023-07-06T01:19:26.631594+0000","last_became_active":"2023-07-06T01:19:25.650810+0000","last_became_peered":"2023-07-06T01:19:25.650810+0000","last_unstale":"2023-07-06T01:19:26.631594+0000","last_undegraded":"2023-07-06T01:19:26.631594+0000","last_fullsized":"2023-07-06T01:19:26.631594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:39.980297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068611+0000","last_change":"2023-07-06T01:19:25.645030+0000","last_active":"2023-07-06T01:19:27.068611+0000","last_peered":"2023-07-06T01:19:27.068611+0000","last_clean":"2023-07-06T01:19:27.068611+0000","last_became_active":"2023-07-06T01:19:25.644689+0000","last_became_peered":"2023-07-06T01:19:25.644689+0000","last_unstale":"2023-07-06T01:19:27.068611+0000","last_undegraded":"2023-07-06T01:19:27.068611+0000","last_fullsized":"2023-07-06T01:19:27.068611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:36:14.823559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632289+0000","last_change":"2023-07-06T01:19:25.660308+0000","last_active":"2023-07-06T01:19:26.632289+0000","last_peered":"2023-07-06T01:19:26.632289+0000","last_clean":"2023-07-06T01:19:26.632289+0000","last_became_active":"2023-07-06T01:19:25.658245+0000","last_became_peered":"2023-07-06T01:19:25.658245+0000","last_unstale":"2023-07-06T01:19:26.632289+0000","last_undegraded":"2023-07-06T01:19:26.632289+0000","last_fullsized":"2023-07-06T01:19:26.632289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:14.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069296+0000","last_change":"2023-07-06T01:19:25.660004+0000","last_active":"2023-07-06T01:19:27.069296+0000","last_peered":"2023-07-06T01:19:27.069296+0000","last_clean":"2023-07-06T01:19:27.069296+0000","last_became_active":"2023-07-06T01:19:25.659487+0000","last_became_peered":"2023-07-06T01:19:25.659487+0000","last_unstale":"2023-07-06T01:19:27.069296+0000","last_undegraded":"2023-07-06T01:19:27.069296+0000","last_fullsized":"2023-07-06T01:19:27.069296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:14.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630591+0000","last_change":"2023-07-06T01:19:25.636207+0000","last_active":"2023-07-06T01:19:26.630591+0000","last_peered":"2023-07-06T01:19:26.630591+0000","last_clean":"2023-07-06T01:19:26.630591+0000","last_became_active":"2023-07-06T01:19:25.635876+0000","last_became_peered":"2023-07-06T01:19:25.635876+0000","last_unstale":"2023-07-06T01:19:26.630591+0000","last_undegraded":"2023-07-06T01:19:26.630591+0000","last_fullsized":"2023-07-06T01:19:26.630591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:25.693365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944279+0000","last_change":"2023-07-06T01:19:25.655490+0000","last_active":"2023-07-06T01:19:26.944279+0000","last_peered":"2023-07-06T01:19:26.944279+0000","last_clean":"2023-07-06T01:19:26.944279+0000","last_became_active":"2023-07-06T01:19:25.655137+0000","last_became_peered":"2023-07-06T01:19:25.655137+0000","last_unstale":"2023-07-06T01:19:26.944279+0000","last_undegraded":"2023-07-06T01:19:26.944279+0000","last_fullsized":"2023-07-06T01:19:26.944279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:09.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068827+0000","last_change":"2023-07-06T01:19:25.655082+0000","last_active":"2023-07-06T01:19:27.068827+0000","last_peered":"2023-07-06T01:19:27.068827+0000","last_clean":"2023-07-06T01:19:27.068827+0000","last_became_active":"2023-07-06T01:19:25.654779+0000","last_became_peered":"2023-07-06T01:19:25.654779+0000","last_unstale":"2023-07-06T01:19:27.068827+0000","last_undegraded":"2023-07-06T01:19:27.068827+0000","last_fullsized":"2023-07-06T01:19:27.068827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:22.147924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068831+0000","last_change":"2023-07-06T01:19:25.644786+0000","last_active":"2023-07-06T01:19:27.068831+0000","last_peered":"2023-07-06T01:19:27.068831+0000","last_clean":"2023-07-06T01:19:27.068831+0000","last_became_active":"2023-07-06T01:19:25.644482+0000","last_became_peered":"2023-07-06T01:19:25.644482+0000","last_unstale":"2023-07-06T01:19:27.068831+0000","last_undegraded":"2023-07-06T01:19:27.068831+0000","last_fullsized":"2023-07-06T01:19:27.068831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:42.802038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068413+0000","last_change":"2023-07-06T01:19:25.642117+0000","last_active":"2023-07-06T01:19:27.068413+0000","last_peered":"2023-07-06T01:19:27.068413+0000","last_clean":"2023-07-06T01:19:27.068413+0000","last_became_active":"2023-07-06T01:19:25.641830+0000","last_became_peered":"2023-07-06T01:19:25.641830+0000","last_unstale":"2023-07-06T01:19:27.068413+0000","last_undegraded":"2023-07-06T01:19:27.068413+0000","last_fullsized":"2023-07-06T01:19:27.068413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:43:11.310891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631861+0000","last_change":"2023-07-06T01:19:25.656605+0000","last_active":"2023-07-06T01:19:26.631861+0000","last_peered":"2023-07-06T01:19:26.631861+0000","last_clean":"2023-07-06T01:19:26.631861+0000","last_became_active":"2023-07-06T01:19:25.656259+0000","last_became_peered":"2023-07-06T01:19:25.656259+0000","last_unstale":"2023-07-06T01:19:26.631861+0000","last_undegraded":"2023-07-06T01:19:26.631861+0000","last_fullsized":"2023-07-06T01:19:26.631861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:40.213207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631966+0000","last_change":"2023-07-06T01:19:24.646013+0000","last_active":"2023-07-06T01:19:26.631966+0000","last_peered":"2023-07-06T01:19:26.631966+0000","last_clean":"2023-07-06T01:19:26.631966+0000","last_became_active":"2023-07-06T01:19:24.644677+0000","last_became_peered":"2023-07-06T01:19:24.644677+0000","last_unstale":"2023-07-06T01:19:26.631966+0000","last_undegraded":"2023-07-06T01:19:26.631966+0000","last_fullsized":"2023-07-06T01:19:26.631966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:17.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944791+0000","last_change":"2023-07-06T01:19:25.650690+0000","last_active":"2023-07-06T01:19:26.944791+0000","last_peered":"2023-07-06T01:19:26.944791+0000","last_clean":"2023-07-06T01:19:26.944791+0000","last_became_active":"2023-07-06T01:19:25.650316+0000","last_became_peered":"2023-07-06T01:19:25.650316+0000","last_unstale":"2023-07-06T01:19:26.944791+0000","last_undegraded":"2023-07-06T01:19:26.944791+0000","last_fullsized":"2023-07-06T01:19:26.944791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:02.031653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068547+0000","last_change":"2023-07-06T01:19:24.645053+0000","last_active":"2023-07-06T01:19:27.068547+0000","last_peered":"2023-07-06T01:19:27.068547+0000","last_clean":"2023-07-06T01:19:27.068547+0000","last_became_active":"2023-07-06T01:19:24.644766+0000","last_became_peered":"2023-07-06T01:19:24.644766+0000","last_unstale":"2023-07-06T01:19:27.068547+0000","last_undegraded":"2023-07-06T01:19:27.068547+0000","last_fullsized":"2023-07-06T01:19:27.068547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:43.286430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630830+0000","last_change":"2023-07-06T01:19:25.649424+0000","last_active":"2023-07-06T01:19:26.630830+0000","last_peered":"2023-07-06T01:19:26.630830+0000","last_clean":"2023-07-06T01:19:26.630830+0000","last_became_active":"2023-07-06T01:19:25.649164+0000","last_became_peered":"2023-07-06T01:19:25.649164+0000","last_unstale":"2023-07-06T01:19:26.630830+0000","last_undegraded":"2023-07-06T01:19:26.630830+0000","last_fullsized":"2023-07-06T01:19:26.630830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:00.621635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944431+0000","last_change":"2023-07-06T01:19:24.645861+0000","last_active":"2023-07-06T01:19:26.944431+0000","last_peered":"2023-07-06T01:19:26.944431+0000","last_clean":"2023-07-06T01:19:26.944431+0000","last_became_active":"2023-07-06T01:19:24.645583+0000","last_became_peered":"2023-07-06T01:19:24.645583+0000","last_unstale":"2023-07-06T01:19:26.944431+0000","last_undegraded":"2023-07-06T01:19:26.944431+0000","last_fullsized":"2023-07-06T01:19:26.944431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:38.975082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069581+0000","last_change":"2023-07-06T01:19:25.656345+0000","last_active":"2023-07-06T01:19:27.069581+0000","last_peered":"2023-07-06T01:19:27.069581+0000","last_clean":"2023-07-06T01:19:27.069581+0000","last_became_active":"2023-07-06T01:19:25.655955+0000","last_became_peered":"2023-07-06T01:19:25.655955+0000","last_unstale":"2023-07-06T01:19:27.069581+0000","last_undegraded":"2023-07-06T01:19:27.069581+0000","last_fullsized":"2023-07-06T01:19:27.069581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:45.855998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630810+0000","last_change":"2023-07-06T01:19:24.643151+0000","last_active":"2023-07-06T01:19:26.630810+0000","last_peered":"2023-07-06T01:19:26.630810+0000","last_clean":"2023-07-06T01:19:26.630810+0000","last_became_active":"2023-07-06T01:19:24.642827+0000","last_became_peered":"2023-07-06T01:19:24.642827+0000","last_unstale":"2023-07-06T01:19:26.630810+0000","last_undegraded":"2023-07-06T01:19:26.630810+0000","last_fullsized":"2023-07-06T01:19:26.630810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:08.965590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944402+0000","last_change":"2023-07-06T01:19:25.652005+0000","last_active":"2023-07-06T01:19:26.944402+0000","last_peered":"2023-07-06T01:19:26.944402+0000","last_clean":"2023-07-06T01:19:26.944402+0000","last_became_active":"2023-07-06T01:19:25.651299+0000","last_became_peered":"2023-07-06T01:19:25.651299+0000","last_unstale":"2023-07-06T01:19:26.944402+0000","last_undegraded":"2023-07-06T01:19:26.944402+0000","last_fullsized":"2023-07-06T01:19:26.944402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632387+0000","last_change":"2023-07-06T01:19:24.642960+0000","last_active":"2023-07-06T01:19:26.632387+0000","last_peered":"2023-07-06T01:19:26.632387+0000","last_clean":"2023-07-06T01:19:26.632387+0000","last_became_active":"2023-07-06T01:19:24.642667+0000","last_became_peered":"2023-07-06T01:19:24.642667+0000","last_unstale":"2023-07-06T01:19:26.632387+0000","last_undegraded":"2023-07-06T01:19:26.632387+0000","last_fullsized":"2023-07-06T01:19:26.632387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:30.566175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944596+0000","last_change":"2023-07-06T01:19:25.650426+0000","last_active":"2023-07-06T01:19:26.944596+0000","last_peered":"2023-07-06T01:19:26.944596+0000","last_clean":"2023-07-06T01:19:26.944596+0000","last_became_active":"2023-07-06T01:19:25.650130+0000","last_became_peered":"2023-07-06T01:19:25.650130+0000","last_unstale":"2023-07-06T01:19:26.944596+0000","last_undegraded":"2023-07-06T01:19:26.944596+0000","last_fullsized":"2023-07-06T01:19:26.944596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:45:08.937322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069529+0000","last_change":"2023-07-06T01:19:24.644753+0000","last_active":"2023-07-06T01:19:27.069529+0000","last_peered":"2023-07-06T01:19:27.069529+0000","last_clean":"2023-07-06T01:19:27.069529+0000","last_became_active":"2023-07-06T01:19:24.644420+0000","last_became_peered":"2023-07-06T01:19:24.644420+0000","last_unstale":"2023-07-06T01:19:27.069529+0000","last_undegraded":"2023-07-06T01:19:27.069529+0000","last_fullsized":"2023-07-06T01:19:27.069529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:01.026438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068483+0000","last_change":"2023-07-06T01:19:25.652623+0000","last_active":"2023-07-06T01:19:27.068483+0000","last_peered":"2023-07-06T01:19:27.068483+0000","last_clean":"2023-07-06T01:19:27.068483+0000","last_became_active":"2023-07-06T01:19:25.652341+0000","last_became_peered":"2023-07-06T01:19:25.652341+0000","last_unstale":"2023-07-06T01:19:27.068483+0000","last_undegraded":"2023-07-06T01:19:27.068483+0000","last_fullsized":"2023-07-06T01:19:27.068483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:21:00.969228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632583+0000","last_change":"2023-07-06T01:19:24.648863+0000","last_active":"2023-07-06T01:19:26.632583+0000","last_peered":"2023-07-06T01:19:26.632583+0000","last_clean":"2023-07-06T01:19:26.632583+0000","last_became_active":"2023-07-06T01:19:24.648520+0000","last_became_peered":"2023-07-06T01:19:24.648520+0000","last_unstale":"2023-07-06T01:19:26.632583+0000","last_undegraded":"2023-07-06T01:19:26.632583+0000","last_fullsized":"2023-07-06T01:19:26.632583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:31:21.451680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632381+0000","last_change":"2023-07-06T01:19:25.653993+0000","last_active":"2023-07-06T01:19:26.632381+0000","last_peered":"2023-07-06T01:19:26.632381+0000","last_clean":"2023-07-06T01:19:26.632381+0000","last_became_active":"2023-07-06T01:19:25.653093+0000","last_became_peered":"2023-07-06T01:19:25.653093+0000","last_unstale":"2023-07-06T01:19:26.632381+0000","last_undegraded":"2023-07-06T01:19:26.632381+0000","last_fullsized":"2023-07-06T01:19:26.632381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:02.031653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069278+0000","last_change":"2023-07-06T01:19:24.651048+0000","last_active":"2023-07-06T01:19:27.069278+0000","last_peered":"2023-07-06T01:19:27.069278+0000","last_clean":"2023-07-06T01:19:27.069278+0000","last_became_active":"2023-07-06T01:19:24.650735+0000","last_became_peered":"2023-07-06T01:19:24.650735+0000","last_unstale":"2023-07-06T01:19:27.069278+0000","last_undegraded":"2023-07-06T01:19:27.069278+0000","last_fullsized":"2023-07-06T01:19:27.069278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:58.274250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944680+0000","last_change":"2023-07-06T01:19:25.651034+0000","last_active":"2023-07-06T01:19:26.944680+0000","last_peered":"2023-07-06T01:19:26.944680+0000","last_clean":"2023-07-06T01:19:26.944680+0000","last_became_active":"2023-07-06T01:19:25.650753+0000","last_became_peered":"2023-07-06T01:19:25.650753+0000","last_unstale":"2023-07-06T01:19:26.944680+0000","last_undegraded":"2023-07-06T01:19:26.944680+0000","last_fullsized":"2023-07-06T01:19:26.944680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:36.632721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630793+0000","last_change":"2023-07-06T01:19:24.648974+0000","last_active":"2023-07-06T01:19:26.630793+0000","last_peered":"2023-07-06T01:19:26.630793+0000","last_clean":"2023-07-06T01:19:26.630793+0000","last_became_active":"2023-07-06T01:19:24.648691+0000","last_became_peered":"2023-07-06T01:19:24.648691+0000","last_unstale":"2023-07-06T01:19:26.630793+0000","last_undegraded":"2023-07-06T01:19:26.630793+0000","last_fullsized":"2023-07-06T01:19:26.630793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:44.850783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631977+0000","last_change":"2023-07-06T01:19:25.656530+0000","last_active":"2023-07-06T01:19:26.631977+0000","last_peered":"2023-07-06T01:19:26.631977+0000","last_clean":"2023-07-06T01:19:26.631977+0000","last_became_active":"2023-07-06T01:19:25.655966+0000","last_became_peered":"2023-07-06T01:19:25.655966+0000","last_unstale":"2023-07-06T01:19:26.631977+0000","last_undegraded":"2023-07-06T01:19:26.631977+0000","last_fullsized":"2023-07-06T01:19:26.631977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:10.760963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069121+0000","last_change":"2023-07-06T01:19:24.649133+0000","last_active":"2023-07-06T01:19:27.069121+0000","last_peered":"2023-07-06T01:19:27.069121+0000","last_clean":"2023-07-06T01:19:27.069121+0000","last_became_active":"2023-07-06T01:19:24.648818+0000","last_became_peered":"2023-07-06T01:19:24.648818+0000","last_unstale":"2023-07-06T01:19:27.069121+0000","last_undegraded":"2023-07-06T01:19:27.069121+0000","last_fullsized":"2023-07-06T01:19:27.069121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:45:07.932107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630689+0000","last_change":"2023-07-06T01:19:25.646210+0000","last_active":"2023-07-06T01:19:26.630689+0000","last_peered":"2023-07-06T01:19:26.630689+0000","last_clean":"2023-07-06T01:19:26.630689+0000","last_became_active":"2023-07-06T01:19:25.645912+0000","last_became_peered":"2023-07-06T01:19:25.645912+0000","last_unstale":"2023-07-06T01:19:26.630689+0000","last_undegraded":"2023-07-06T01:19:26.630689+0000","last_fullsized":"2023-07-06T01:19:26.630689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:21.566043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068646+0000","last_change":"2023-07-06T01:19:24.642544+0000","last_active":"2023-07-06T01:19:27.068646+0000","last_peered":"2023-07-06T01:19:27.068646+0000","last_clean":"2023-07-06T01:19:27.068646+0000","last_became_active":"2023-07-06T01:19:24.642267+0000","last_became_peered":"2023-07-06T01:19:24.642267+0000","last_unstale":"2023-07-06T01:19:27.068646+0000","last_undegraded":"2023-07-06T01:19:27.068646+0000","last_fullsized":"2023-07-06T01:19:27.068646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:20:59.964013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944798+0000","last_change":"2023-07-06T01:19:25.650388+0000","last_active":"2023-07-06T01:19:26.944798+0000","last_peered":"2023-07-06T01:19:26.944798+0000","last_clean":"2023-07-06T01:19:26.944798+0000","last_became_active":"2023-07-06T01:19:25.650047+0000","last_became_peered":"2023-07-06T01:19:25.650047+0000","last_unstale":"2023-07-06T01:19:26.944798+0000","last_undegraded":"2023-07-06T01:19:26.944798+0000","last_fullsized":"2023-07-06T01:19:26.944798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:08:02.466103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069262+0000","last_change":"2023-07-06T01:19:24.647357+0000","last_active":"2023-07-06T01:19:27.069262+0000","last_peered":"2023-07-06T01:19:27.069262+0000","last_clean":"2023-07-06T01:19:27.069262+0000","last_became_active":"2023-07-06T01:19:24.647103+0000","last_became_peered":"2023-07-06T01:19:24.647103+0000","last_unstale":"2023-07-06T01:19:27.069262+0000","last_undegraded":"2023-07-06T01:19:27.069262+0000","last_fullsized":"2023-07-06T01:19:27.069262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:05:35.627505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631268+0000","last_change":"2023-07-06T01:19:25.649637+0000","last_active":"2023-07-06T01:19:26.631268+0000","last_peered":"2023-07-06T01:19:26.631268+0000","last_clean":"2023-07-06T01:19:26.631268+0000","last_became_active":"2023-07-06T01:19:25.649363+0000","last_became_peered":"2023-07-06T01:19:25.649363+0000","last_unstale":"2023-07-06T01:19:26.631268+0000","last_undegraded":"2023-07-06T01:19:26.631268+0000","last_fullsized":"2023-07-06T01:19:26.631268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:44.291645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069101+0000","last_change":"2023-07-06T01:19:24.647245+0000","last_active":"2023-07-06T01:19:27.069101+0000","last_peered":"2023-07-06T01:19:27.069101+0000","last_clean":"2023-07-06T01:19:27.069101+0000","last_became_active":"2023-07-06T01:19:24.646993+0000","last_became_peered":"2023-07-06T01:19:24.646993+0000","last_unstale":"2023-07-06T01:19:27.069101+0000","last_undegraded":"2023-07-06T01:19:27.069101+0000","last_fullsized":"2023-07-06T01:19:27.069101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:17.214380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631382+0000","last_change":"2023-07-06T01:19:25.657736+0000","last_active":"2023-07-06T01:19:26.631382+0000","last_peered":"2023-07-06T01:19:26.631382+0000","last_clean":"2023-07-06T01:19:26.631382+0000","last_became_active":"2023-07-06T01:19:25.657511+0000","last_became_peered":"2023-07-06T01:19:25.657511+0000","last_unstale":"2023-07-06T01:19:26.631382+0000","last_undegraded":"2023-07-06T01:19:26.631382+0000","last_fullsized":"2023-07-06T01:19:26.631382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:15:00.621635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069462+0000","last_change":"2023-07-06T01:19:24.647904+0000","last_active":"2023-07-06T01:19:27.069462+0000","last_peered":"2023-07-06T01:19:27.069462+0000","last_clean":"2023-07-06T01:19:27.069462+0000","last_became_active":"2023-07-06T01:19:24.647621+0000","last_became_peered":"2023-07-06T01:19:24.647621+0000","last_unstale":"2023-07-06T01:19:27.069462+0000","last_undegraded":"2023-07-06T01:19:27.069462+0000","last_fullsized":"2023-07-06T01:19:27.069462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:08:01.460887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069126+0000","last_change":"2023-07-06T01:19:25.652728+0000","last_active":"2023-07-06T01:19:27.069126+0000","last_peered":"2023-07-06T01:19:27.069126+0000","last_clean":"2023-07-06T01:19:27.069126+0000","last_became_active":"2023-07-06T01:19:25.652420+0000","last_became_peered":"2023-07-06T01:19:25.652420+0000","last_unstale":"2023-07-06T01:19:27.069126+0000","last_undegraded":"2023-07-06T01:19:27.069126+0000","last_fullsized":"2023-07-06T01:19:27.069126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:40.213207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631482+0000","last_change":"2023-07-06T01:19:24.640722+0000","last_active":"2023-07-06T01:19:26.631482+0000","last_peered":"2023-07-06T01:19:26.631482+0000","last_clean":"2023-07-06T01:19:26.631482+0000","last_became_active":"2023-07-06T01:19:24.640417+0000","last_became_peered":"2023-07-06T01:19:24.640417+0000","last_unstale":"2023-07-06T01:19:26.631482+0000","last_undegraded":"2023-07-06T01:19:26.631482+0000","last_fullsized":"2023-07-06T01:19:26.631482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:24.688149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630955+0000","last_change":"2023-07-06T01:19:25.636624+0000","last_active":"2023-07-06T01:19:26.630955+0000","last_peered":"2023-07-06T01:19:26.630955+0000","last_clean":"2023-07-06T01:19:26.630955+0000","last_became_active":"2023-07-06T01:19:25.636320+0000","last_became_peered":"2023-07-06T01:19:25.636320+0000","last_unstale":"2023-07-06T01:19:26.630955+0000","last_undegraded":"2023-07-06T01:19:26.630955+0000","last_fullsized":"2023-07-06T01:19:26.630955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944677+0000","last_change":"2023-07-06T01:19:24.645728+0000","last_active":"2023-07-06T01:19:26.944677+0000","last_peered":"2023-07-06T01:19:26.944677+0000","last_clean":"2023-07-06T01:19:26.944677+0000","last_became_active":"2023-07-06T01:19:24.644479+0000","last_became_peered":"2023-07-06T01:19:24.644479+0000","last_unstale":"2023-07-06T01:19:26.944677+0000","last_undegraded":"2023-07-06T01:19:26.944677+0000","last_fullsized":"2023-07-06T01:19:26.944677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:58.274250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632444+0000","last_change":"2023-07-06T01:19:25.658759+0000","last_active":"2023-07-06T01:19:26.632444+0000","last_peered":"2023-07-06T01:19:26.632444+0000","last_clean":"2023-07-06T01:19:26.632444+0000","last_became_active":"2023-07-06T01:19:25.658482+0000","last_became_peered":"2023-07-06T01:19:25.658482+0000","last_unstale":"2023-07-06T01:19:26.632444+0000","last_undegraded":"2023-07-06T01:19:26.632444+0000","last_fullsized":"2023-07-06T01:19:26.632444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:58:41.885622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069265+0000","last_change":"2023-07-06T01:19:24.649526+0000","last_active":"2023-07-06T01:19:27.069265+0000","last_peered":"2023-07-06T01:19:27.069265+0000","last_clean":"2023-07-06T01:19:27.069265+0000","last_became_active":"2023-07-06T01:19:24.648970+0000","last_became_peered":"2023-07-06T01:19:24.648970+0000","last_unstale":"2023-07-06T01:19:27.069265+0000","last_undegraded":"2023-07-06T01:19:27.069265+0000","last_fullsized":"2023-07-06T01:19:27.069265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:41.796823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632518+0000","last_change":"2023-07-06T01:19:25.648610+0000","last_active":"2023-07-06T01:19:26.632518+0000","last_peered":"2023-07-06T01:19:26.632518+0000","last_clean":"2023-07-06T01:19:26.632518+0000","last_became_active":"2023-07-06T01:19:25.648317+0000","last_became_peered":"2023-07-06T01:19:25.648317+0000","last_unstale":"2023-07-06T01:19:26.632518+0000","last_undegraded":"2023-07-06T01:19:26.632518+0000","last_fullsized":"2023-07-06T01:19:26.632518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:47:36.660557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631774+0000","last_change":"2023-07-06T01:19:24.641218+0000","last_active":"2023-07-06T01:19:26.631774+0000","last_peered":"2023-07-06T01:19:26.631774+0000","last_clean":"2023-07-06T01:19:26.631774+0000","last_became_active":"2023-07-06T01:19:24.640780+0000","last_became_peered":"2023-07-06T01:19:24.640780+0000","last_unstale":"2023-07-06T01:19:26.631774+0000","last_undegraded":"2023-07-06T01:19:26.631774+0000","last_fullsized":"2023-07-06T01:19:26.631774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:21.142709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631764+0000","last_change":"2023-07-06T01:19:25.657935+0000","last_active":"2023-07-06T01:19:26.631764+0000","last_peered":"2023-07-06T01:19:26.631764+0000","last_clean":"2023-07-06T01:19:26.631764+0000","last_became_active":"2023-07-06T01:19:25.653472+0000","last_became_peered":"2023-07-06T01:19:25.653472+0000","last_unstale":"2023-07-06T01:19:26.631764+0000","last_undegraded":"2023-07-06T01:19:26.631764+0000","last_fullsized":"2023-07-06T01:19:26.631764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:45:08.937322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944291+0000","last_change":"2023-07-06T01:19:24.645128+0000","last_active":"2023-07-06T01:19:26.944291+0000","last_peered":"2023-07-06T01:19:26.944291+0000","last_clean":"2023-07-06T01:19:26.944291+0000","last_became_active":"2023-07-06T01:19:24.644825+0000","last_became_peered":"2023-07-06T01:19:24.644825+0000","last_unstale":"2023-07-06T01:19:26.944291+0000","last_undegraded":"2023-07-06T01:19:26.944291+0000","last_fullsized":"2023-07-06T01:19:26.944291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:14:59.616420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069000+0000","last_change":"2023-07-06T01:19:25.654566+0000","last_active":"2023-07-06T01:19:27.069000+0000","last_peered":"2023-07-06T01:19:27.069000+0000","last_clean":"2023-07-06T01:19:27.069000+0000","last_became_active":"2023-07-06T01:19:25.654238+0000","last_became_peered":"2023-07-06T01:19:25.654238+0000","last_unstale":"2023-07-06T01:19:27.069000+0000","last_undegraded":"2023-07-06T01:19:27.069000+0000","last_fullsized":"2023-07-06T01:19:27.069000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:45.855998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944614+0000","last_change":"2023-07-06T01:19:24.648921+0000","last_active":"2023-07-06T01:19:26.944614+0000","last_peered":"2023-07-06T01:19:26.944614+0000","last_clean":"2023-07-06T01:19:26.944614+0000","last_became_active":"2023-07-06T01:19:24.648676+0000","last_became_peered":"2023-07-06T01:19:24.648676+0000","last_unstale":"2023-07-06T01:19:26.944614+0000","last_undegraded":"2023-07-06T01:19:26.944614+0000","last_fullsized":"2023-07-06T01:19:26.944614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:41:33.528500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632266+0000","last_change":"2023-07-06T01:19:20.074195+0000","last_active":"2023-07-06T01:19:26.632266+0000","last_peered":"2023-07-06T01:19:26.632266+0000","last_clean":"2023-07-06T01:19:26.632266+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:19:26.632266+0000","last_undegraded":"2023-07-06T01:19:26.632266+0000","last_fullsized":"2023-07-06T01:19:26.632266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:07:56.411730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944214+0000","last_change":"2023-07-06T01:19:25.652132+0000","last_active":"2023-07-06T01:19:26.944214+0000","last_peered":"2023-07-06T01:19:26.944214+0000","last_clean":"2023-07-06T01:19:26.944214+0000","last_became_active":"2023-07-06T01:19:25.651678+0000","last_became_peered":"2023-07-06T01:19:25.651678+0000","last_unstale":"2023-07-06T01:19:26.944214+0000","last_undegraded":"2023-07-06T01:19:26.944214+0000","last_fullsized":"2023-07-06T01:19:26.944214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:45.855998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944468+0000","last_change":"2023-07-06T01:19:19.587110+0000","last_active":"2023-07-06T01:19:26.944468+0000","last_peered":"2023-07-06T01:19:26.944468+0000","last_clean":"2023-07-06T01:19:26.944468+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:19:26.944468+0000","last_undegraded":"2023-07-06T01:19:26.944468+0000","last_fullsized":"2023-07-06T01:19:26.944468+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:23:12.165222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.166192+0000","last_change":"2023-07-06T01:19:24.640259+0000","last_active":"2023-07-06T01:19:27.166192+0000","last_peered":"2023-07-06T01:19:27.166192+0000","last_clean":"2023-07-06T01:19:27.166192+0000","last_became_active":"2023-07-06T01:19:24.639942+0000","last_became_peered":"2023-07-06T01:19:24.639942+0000","last_unstale":"2023-07-06T01:19:27.166192+0000","last_undegraded":"2023-07-06T01:19:27.166192+0000","last_fullsized":"2023-07-06T01:19:27.166192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:30.566175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631522+0000","last_change":"2023-07-06T01:19:25.657104+0000","last_active":"2023-07-06T01:19:26.631522+0000","last_peered":"2023-07-06T01:19:26.631522+0000","last_clean":"2023-07-06T01:19:26.631522+0000","last_became_active":"2023-07-06T01:19:25.656804+0000","last_became_peered":"2023-07-06T01:19:25.656804+0000","last_unstale":"2023-07-06T01:19:26.631522+0000","last_undegraded":"2023-07-06T01:19:26.631522+0000","last_fullsized":"2023-07-06T01:19:26.631522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:36.762610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068648+0000","last_change":"2023-07-06T01:19:19.584242+0000","last_active":"2023-07-06T01:19:27.068648+0000","last_peered":"2023-07-06T01:19:27.068648+0000","last_clean":"2023-07-06T01:19:27.068648+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:19:27.068648+0000","last_undegraded":"2023-07-06T01:19:27.068648+0000","last_fullsized":"2023-07-06T01:19:27.068648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:27.113987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":169,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068934+0000","last_change":"2023-07-06T01:19:19.586495+0000","last_active":"2023-07-06T01:19:27.068934+0000","last_peered":"2023-07-06T01:19:27.068934+0000","last_clean":"2023-07-06T01:19:27.068934+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:19:27.068934+0000","last_undegraded":"2023-07-06T01:19:27.068934+0000","last_fullsized":"2023-07-06T01:19:27.068934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:01:53.085093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630456+0000","last_change":"2023-07-06T01:19:24.642436+0000","last_active":"2023-07-06T01:19:26.630456+0000","last_peered":"2023-07-06T01:19:26.630456+0000","last_clean":"2023-07-06T01:19:26.630456+0000","last_became_active":"2023-07-06T01:19:24.641979+0000","last_became_peered":"2023-07-06T01:19:24.641979+0000","last_unstale":"2023-07-06T01:19:26.630456+0000","last_undegraded":"2023-07-06T01:19:26.630456+0000","last_fullsized":"2023-07-06T01:19:26.630456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:43.286430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630728+0000","last_change":"2023-07-06T01:19:25.639313+0000","last_active":"2023-07-06T01:19:26.630728+0000","last_peered":"2023-07-06T01:19:26.630728+0000","last_clean":"2023-07-06T01:19:26.630728+0000","last_became_active":"2023-07-06T01:19:25.639042+0000","last_became_peered":"2023-07-06T01:19:25.639042+0000","last_unstale":"2023-07-06T01:19:26.630728+0000","last_undegraded":"2023-07-06T01:19:26.630728+0000","last_fullsized":"2023-07-06T01:19:26.630728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:40.213207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068698+0000","last_change":"2023-07-06T01:19:19.586426+0000","last_active":"2023-07-06T01:19:27.068698+0000","last_peered":"2023-07-06T01:19:27.068698+0000","last_clean":"2023-07-06T01:19:27.068698+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:19:27.068698+0000","last_undegraded":"2023-07-06T01:19:27.068698+0000","last_fullsized":"2023-07-06T01:19:27.068698+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:47:33.925924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068473+0000","last_change":"2023-07-06T01:19:24.643437+0000","last_active":"2023-07-06T01:19:27.068473+0000","last_peered":"2023-07-06T01:19:27.068473+0000","last_clean":"2023-07-06T01:19:27.068473+0000","last_became_active":"2023-07-06T01:19:24.642926+0000","last_became_peered":"2023-07-06T01:19:24.642926+0000","last_unstale":"2023-07-06T01:19:27.068473+0000","last_undegraded":"2023-07-06T01:19:27.068473+0000","last_fullsized":"2023-07-06T01:19:27.068473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:17.543019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068999+0000","last_change":"2023-07-06T01:19:25.656081+0000","last_active":"2023-07-06T01:19:27.068999+0000","last_peered":"2023-07-06T01:19:27.068999+0000","last_clean":"2023-07-06T01:19:27.068999+0000","last_became_active":"2023-07-06T01:19:25.655801+0000","last_became_peered":"2023-07-06T01:19:25.655801+0000","last_unstale":"2023-07-06T01:19:27.068999+0000","last_undegraded":"2023-07-06T01:19:27.068999+0000","last_fullsized":"2023-07-06T01:19:27.068999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:10.760963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068626+0000","last_change":"2023-07-06T01:19:19.586626+0000","last_active":"2023-07-06T01:19:27.068626+0000","last_peered":"2023-07-06T01:19:27.068626+0000","last_clean":"2023-07-06T01:19:27.068626+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:19:27.068626+0000","last_undegraded":"2023-07-06T01:19:27.068626+0000","last_fullsized":"2023-07-06T01:19:27.068626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:13:30.708237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944310+0000","last_change":"2023-07-06T01:19:24.644389+0000","last_active":"2023-07-06T01:19:26.944310+0000","last_peered":"2023-07-06T01:19:26.944310+0000","last_clean":"2023-07-06T01:19:26.944310+0000","last_became_active":"2023-07-06T01:19:24.644113+0000","last_became_peered":"2023-07-06T01:19:24.644113+0000","last_unstale":"2023-07-06T01:19:26.944310+0000","last_undegraded":"2023-07-06T01:19:26.944310+0000","last_fullsized":"2023-07-06T01:19:26.944310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:48:30.566175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069023+0000","last_change":"2023-07-06T01:19:25.654127+0000","last_active":"2023-07-06T01:19:27.069023+0000","last_peered":"2023-07-06T01:19:27.069023+0000","last_clean":"2023-07-06T01:19:27.069023+0000","last_became_active":"2023-07-06T01:19:25.653903+0000","last_became_peered":"2023-07-06T01:19:25.653903+0000","last_unstale":"2023-07-06T01:19:27.069023+0000","last_undegraded":"2023-07-06T01:19:27.069023+0000","last_fullsized":"2023-07-06T01:19:27.069023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:25.693365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631434+0000","last_change":"2023-07-06T01:19:19.584086+0000","last_active":"2023-07-06T01:19:26.631434+0000","last_peered":"2023-07-06T01:19:26.631434+0000","last_clean":"2023-07-06T01:19:26.631434+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:19:26.631434+0000","last_undegraded":"2023-07-06T01:19:26.631434+0000","last_fullsized":"2023-07-06T01:19:26.631434+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:28:36.747665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632572+0000","last_change":"2023-07-06T01:19:24.648405+0000","last_active":"2023-07-06T01:19:26.632572+0000","last_peered":"2023-07-06T01:19:26.632572+0000","last_clean":"2023-07-06T01:19:26.632572+0000","last_became_active":"2023-07-06T01:19:24.648161+0000","last_became_peered":"2023-07-06T01:19:24.648161+0000","last_unstale":"2023-07-06T01:19:26.632572+0000","last_undegraded":"2023-07-06T01:19:26.632572+0000","last_fullsized":"2023-07-06T01:19:26.632572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:39:24.607769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069089+0000","last_change":"2023-07-06T01:19:25.641885+0000","last_active":"2023-07-06T01:19:27.069089+0000","last_peered":"2023-07-06T01:19:27.069089+0000","last_clean":"2023-07-06T01:19:27.069089+0000","last_became_active":"2023-07-06T01:19:25.641570+0000","last_became_peered":"2023-07-06T01:19:25.641570+0000","last_unstale":"2023-07-06T01:19:27.069089+0000","last_undegraded":"2023-07-06T01:19:27.069089+0000","last_fullsized":"2023-07-06T01:19:27.069089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:09.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069072+0000","last_change":"2023-07-06T01:19:19.586424+0000","last_active":"2023-07-06T01:19:27.069072+0000","last_peered":"2023-07-06T01:19:27.069072+0000","last_clean":"2023-07-06T01:19:27.069072+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:19:27.069072+0000","last_undegraded":"2023-07-06T01:19:27.069072+0000","last_fullsized":"2023-07-06T01:19:27.069072+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:15.511670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068948+0000","last_change":"2023-07-06T01:19:24.644635+0000","last_active":"2023-07-06T01:19:27.068948+0000","last_peered":"2023-07-06T01:19:27.068948+0000","last_clean":"2023-07-06T01:19:27.068948+0000","last_became_active":"2023-07-06T01:19:24.644344+0000","last_became_peered":"2023-07-06T01:19:24.644344+0000","last_unstale":"2023-07-06T01:19:27.068948+0000","last_undegraded":"2023-07-06T01:19:27.068948+0000","last_fullsized":"2023-07-06T01:19:27.068948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:03:39.207991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631123+0000","last_change":"2023-07-06T01:19:25.657020+0000","last_active":"2023-07-06T01:19:26.631123+0000","last_peered":"2023-07-06T01:19:26.631123+0000","last_clean":"2023-07-06T01:19:26.631123+0000","last_became_active":"2023-07-06T01:19:25.656084+0000","last_became_peered":"2023-07-06T01:19:25.656084+0000","last_unstale":"2023-07-06T01:19:26.631123+0000","last_undegraded":"2023-07-06T01:19:26.631123+0000","last_fullsized":"2023-07-06T01:19:26.631123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:22.147924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069477+0000","last_change":"2023-07-06T01:19:19.583699+0000","last_active":"2023-07-06T01:19:27.069477+0000","last_peered":"2023-07-06T01:19:27.069477+0000","last_clean":"2023-07-06T01:19:27.069477+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:19:27.069477+0000","last_undegraded":"2023-07-06T01:19:27.069477+0000","last_fullsized":"2023-07-06T01:19:27.069477+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:04.706590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631992+0000","last_change":"2023-07-06T01:19:24.642998+0000","last_active":"2023-07-06T01:19:26.631992+0000","last_peered":"2023-07-06T01:19:26.631992+0000","last_clean":"2023-07-06T01:19:26.631992+0000","last_became_active":"2023-07-06T01:19:24.642674+0000","last_became_peered":"2023-07-06T01:19:24.642674+0000","last_unstale":"2023-07-06T01:19:26.631992+0000","last_undegraded":"2023-07-06T01:19:26.631992+0000","last_fullsized":"2023-07-06T01:19:26.631992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:36:44.850783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068822+0000","last_change":"2023-07-06T01:19:25.651577+0000","last_active":"2023-07-06T01:19:27.068822+0000","last_peered":"2023-07-06T01:19:27.068822+0000","last_clean":"2023-07-06T01:19:27.068822+0000","last_became_active":"2023-07-06T01:19:25.651299+0000","last_became_peered":"2023-07-06T01:19:25.651299+0000","last_unstale":"2023-07-06T01:19:27.068822+0000","last_undegraded":"2023-07-06T01:19:27.068822+0000","last_fullsized":"2023-07-06T01:19:27.068822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:25.693365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631194+0000","last_change":"2023-07-06T01:19:24.648687+0000","last_active":"2023-07-06T01:19:26.631194+0000","last_peered":"2023-07-06T01:19:26.631194+0000","last_clean":"2023-07-06T01:19:26.631194+0000","last_became_active":"2023-07-06T01:19:24.648363+0000","last_became_peered":"2023-07-06T01:19:24.648363+0000","last_unstale":"2023-07-06T01:19:26.631194+0000","last_undegraded":"2023-07-06T01:19:26.631194+0000","last_fullsized":"2023-07-06T01:19:26.631194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:24.719467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.069057+0000","last_change":"2023-07-06T01:19:25.646102+0000","last_active":"2023-07-06T01:19:27.069057+0000","last_peered":"2023-07-06T01:19:27.069057+0000","last_clean":"2023-07-06T01:19:27.069057+0000","last_became_active":"2023-07-06T01:19:25.645911+0000","last_became_peered":"2023-07-06T01:19:25.645911+0000","last_unstale":"2023-07-06T01:19:27.069057+0000","last_undegraded":"2023-07-06T01:19:27.069057+0000","last_fullsized":"2023-07-06T01:19:27.069057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:58:05.681285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631273+0000","last_change":"2023-07-06T01:19:24.645915+0000","last_active":"2023-07-06T01:19:26.631273+0000","last_peered":"2023-07-06T01:19:26.631273+0000","last_clean":"2023-07-06T01:19:26.631273+0000","last_became_active":"2023-07-06T01:19:24.645626+0000","last_became_peered":"2023-07-06T01:19:24.645626+0000","last_unstale":"2023-07-06T01:19:26.631273+0000","last_undegraded":"2023-07-06T01:19:26.631273+0000","last_fullsized":"2023-07-06T01:19:26.631273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:32.163145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632582+0000","last_change":"2023-07-06T01:19:25.658031+0000","last_active":"2023-07-06T01:19:26.632582+0000","last_peered":"2023-07-06T01:19:26.632582+0000","last_clean":"2023-07-06T01:19:26.632582+0000","last_became_active":"2023-07-06T01:19:25.657733+0000","last_became_peered":"2023-07-06T01:19:25.657733+0000","last_unstale":"2023-07-06T01:19:26.632582+0000","last_undegraded":"2023-07-06T01:19:26.632582+0000","last_fullsized":"2023-07-06T01:19:26.632582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:35:30.587396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632124+0000","last_change":"2023-07-06T01:19:24.645784+0000","last_active":"2023-07-06T01:19:26.632124+0000","last_peered":"2023-07-06T01:19:26.632124+0000","last_clean":"2023-07-06T01:19:26.632124+0000","last_became_active":"2023-07-06T01:19:24.645447+0000","last_became_peered":"2023-07-06T01:19:24.645447+0000","last_unstale":"2023-07-06T01:19:26.632124+0000","last_undegraded":"2023-07-06T01:19:26.632124+0000","last_fullsized":"2023-07-06T01:19:26.632124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:08.965590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.632116+0000","last_change":"2023-07-06T01:19:25.653462+0000","last_active":"2023-07-06T01:19:26.632116+0000","last_peered":"2023-07-06T01:19:26.632116+0000","last_clean":"2023-07-06T01:19:26.632116+0000","last_became_active":"2023-07-06T01:19:25.652772+0000","last_became_peered":"2023-07-06T01:19:25.652772+0000","last_unstale":"2023-07-06T01:19:26.632116+0000","last_undegraded":"2023-07-06T01:19:26.632116+0000","last_fullsized":"2023-07-06T01:19:26.632116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:01:59.139466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944733+0000","last_change":"2023-07-06T01:19:24.642392+0000","last_active":"2023-07-06T01:19:26.944733+0000","last_peered":"2023-07-06T01:19:26.944733+0000","last_clean":"2023-07-06T01:19:26.944733+0000","last_became_active":"2023-07-06T01:19:24.642122+0000","last_became_peered":"2023-07-06T01:19:24.642122+0000","last_unstale":"2023-07-06T01:19:26.944733+0000","last_undegraded":"2023-07-06T01:19:26.944733+0000","last_fullsized":"2023-07-06T01:19:26.944733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:01:58.134250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944450+0000","last_change":"2023-07-06T01:19:25.650007+0000","last_active":"2023-07-06T01:19:26.944450+0000","last_peered":"2023-07-06T01:19:26.944450+0000","last_clean":"2023-07-06T01:19:26.944450+0000","last_became_active":"2023-07-06T01:19:25.649108+0000","last_became_peered":"2023-07-06T01:19:25.649108+0000","last_unstale":"2023-07-06T01:19:26.944450+0000","last_undegraded":"2023-07-06T01:19:26.944450+0000","last_fullsized":"2023-07-06T01:19:26.944450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:18:22.147924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.630751+0000","last_change":"2023-07-06T01:19:24.642694+0000","last_active":"2023-07-06T01:19:26.630751+0000","last_peered":"2023-07-06T01:19:26.630751+0000","last_clean":"2023-07-06T01:19:26.630751+0000","last_became_active":"2023-07-06T01:19:24.642130+0000","last_became_peered":"2023-07-06T01:19:24.642130+0000","last_unstale":"2023-07-06T01:19:26.630751+0000","last_undegraded":"2023-07-06T01:19:26.630751+0000","last_fullsized":"2023-07-06T01:19:26.630751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:23.616180+0000","last_clean_scrub_stamp":"2023-07-06T01:19:23.616180+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:10:09.755748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631214+0000","last_change":"2023-07-06T01:19:25.653965+0000","last_active":"2023-07-06T01:19:26.631214+0000","last_peered":"2023-07-06T01:19:26.631214+0000","last_clean":"2023-07-06T01:19:26.631214+0000","last_became_active":"2023-07-06T01:19:25.653448+0000","last_became_peered":"2023-07-06T01:19:25.653448+0000","last_unstale":"2023-07-06T01:19:26.631214+0000","last_undegraded":"2023-07-06T01:19:26.631214+0000","last_fullsized":"2023-07-06T01:19:26.631214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:30:09.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068713+0000","last_change":"2023-07-06T01:19:25.652413+0000","last_active":"2023-07-06T01:19:27.068713+0000","last_peered":"2023-07-06T01:19:27.068713+0000","last_clean":"2023-07-06T01:19:27.068713+0000","last_became_active":"2023-07-06T01:19:25.652051+0000","last_became_peered":"2023-07-06T01:19:25.652051+0000","last_unstale":"2023-07-06T01:19:27.068713+0000","last_undegraded":"2023-07-06T01:19:27.068713+0000","last_fullsized":"2023-07-06T01:19:27.068713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:44.291645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944695+0000","last_change":"2023-07-06T01:19:25.652994+0000","last_active":"2023-07-06T01:19:26.944695+0000","last_peered":"2023-07-06T01:19:26.944695+0000","last_clean":"2023-07-06T01:19:26.944695+0000","last_became_active":"2023-07-06T01:19:25.652484+0000","last_became_peered":"2023-07-06T01:19:25.652484+0000","last_unstale":"2023-07-06T01:19:26.944695+0000","last_undegraded":"2023-07-06T01:19:26.944695+0000","last_fullsized":"2023-07-06T01:19:26.944695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:30:14.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.631822+0000","last_change":"2023-07-06T01:19:25.656779+0000","last_active":"2023-07-06T01:19:26.631822+0000","last_peered":"2023-07-06T01:19:26.631822+0000","last_clean":"2023-07-06T01:19:26.631822+0000","last_became_active":"2023-07-06T01:19:25.653104+0000","last_became_peered":"2023-07-06T01:19:25.653104+0000","last_unstale":"2023-07-06T01:19:26.631822+0000","last_undegraded":"2023-07-06T01:19:26.631822+0000","last_fullsized":"2023-07-06T01:19:26.631822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:41:34.533716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:26.944583+0000","last_change":"2023-07-06T01:19:25.653857+0000","last_active":"2023-07-06T01:19:26.944583+0000","last_peered":"2023-07-06T01:19:26.944583+0000","last_clean":"2023-07-06T01:19:26.944583+0000","last_became_active":"2023-07-06T01:19:25.652976+0000","last_became_peered":"2023-07-06T01:19:25.652976+0000","last_unstale":"2023-07-06T01:19:26.944583+0000","last_undegraded":"2023-07-06T01:19:26.944583+0000","last_fullsized":"2023-07-06T01:19:26.944583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:24:25.724682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-07-06T01:19:27.068930+0000","last_change":"2023-07-06T01:19:25.647146+0000","last_active":"2023-07-06T01:19:27.068930+0000","last_peered":"2023-07-06T01:19:27.068930+0000","last_clean":"2023-07-06T01:19:27.068930+0000","last_became_active":"2023-07-06T01:19:25.646863+0000","last_became_peered":"2023-07-06T01:19:25.646863+0000","last_unstale":"2023-07-06T01:19:27.068930+0000","last_undegraded":"2023-07-06T01:19:27.068930+0000","last_fullsized":"2023-07-06T01:19:27.068930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:24.621395+0000","last_clean_scrub_stamp":"2023-07-06T01:19:24.621395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:21.566043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":670687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":233472,"data_stored":80159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":806912,"data_stored":668907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5180000000000007}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.963000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.397}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.460000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9890000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.097}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1429999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.2430000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":78539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":78629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":79191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":78539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:19:34.971 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-06T01:19:34.971 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-06T01:19:34.972 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-07-06T01:19:34.972 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T01:19:35.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.125+0000 7f22a9b87700 1 Processor -- start 2023-07-06T01:19:35.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.125+0000 7f22a9b87700 1 -- start start 2023-07-06T01:19:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.126+0000 7f22a9b87700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22a40cfbf0 0x7f22a40cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.126+0000 7f22a9b87700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f22a4070a30 0x7f22a415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.126+0000 7f22a9b87700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a415c090 con 0x7f22a406d4a0 2023-07-06T01:19:35.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.126+0000 7f22a9b87700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a40d0500 con 0x7f22a4070a30 2023-07-06T01:19:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.126+0000 7f22a9b87700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40ce5e0 con 0x7f22a40cfbf0 2023-07-06T01:19:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.126+0000 7f22a2ffd700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f22a4070a30 0x7f22a415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.126+0000 7f22a37fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22a40cfbf0 0x7f22a40cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.126+0000 7f22a3fff700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f22a406d4a0 0x7f22a415d7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38606/0 (socket says 172.21.15.22:38606) 2023-07-06T01:19:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.126+0000 7f22a3fff700 1 -- 172.21.15.22:0/3601751620 learned_addr learned my addr 172.21.15.22:0/3601751620 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.127+0000 7f22a2ffd700 1 -- 172.21.15.22:0/3601751620 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22a40cfbf0 msgr2=0x7f22a40cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:35.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.127+0000 7f22a2ffd700 1 --2- 172.21.15.22:0/3601751620 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22a40cfbf0 0x7f22a40cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.127+0000 7f22a2ffd700 1 -- 172.21.15.22:0/3601751620 >> v1:172.21.15.103:6789/0 conn(0x7f22a406d4a0 legacy=0x7f22a415d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.127+0000 7f22a2ffd700 1 -- 172.21.15.22:0/3601751620 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a415a060 con 0x7f22a4070a30 2023-07-06T01:19:35.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.127+0000 7f22a27fc700 1 -- 172.21.15.22:0/3601751620 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4179286932 0 0) 0x7f22a415c090 con 0x7f22a406d4a0 2023-07-06T01:19:35.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.127+0000 7f22a37fe700 1 --2- 172.21.15.22:0/3601751620 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22a40cfbf0 0x7f22a40cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:35.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.127+0000 7f22a2ffd700 1 --2- 172.21.15.22:0/3601751620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22a4070a30 0x7f22a415d280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f228c009840 tx=0x7f228c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73b58babd8f522a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:35.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.127+0000 7f22a27fc700 1 -- 172.21.15.22:0/3601751620 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c011830 con 0x7f22a4070a30 2023-07-06T01:19:35.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.127+0000 7f22a27fc700 1 -- 172.21.15.22:0/3601751620 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c011990 con 0x7f22a4070a30 2023-07-06T01:19:35.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.127+0000 7f22a27fc700 1 -- 172.21.15.22:0/3601751620 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c011d20 con 0x7f22a4070a30 2023-07-06T01:19:35.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.128+0000 7f22a9b87700 1 -- 172.21.15.22:0/3601751620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22a4070a30 msgr2=0x7f22a415d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.128+0000 7f22a9b87700 1 --2- 172.21.15.22:0/3601751620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22a4070a30 0x7f22a415d280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f228c009840 tx=0x7f228c009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:35.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.128+0000 7f22a9b87700 1 -- 172.21.15.22:0/3601751620 shutdown_connections 2023-07-06T01:19:35.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.128+0000 7f22a9b87700 1 --2- 172.21.15.22:0/3601751620 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22a40cfbf0 0x7f22a40cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.128+0000 7f22a9b87700 1 --2- 172.21.15.22:0/3601751620 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22a4070a30 0x7f22a415d280 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.128+0000 7f22a9b87700 1 -- 172.21.15.22:0/3601751620 >> 172.21.15.22:0/3601751620 conn(0x7f22a40b17b0 msgr2=0x7f22a40b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:35.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.128+0000 7f22a9b87700 1 -- 172.21.15.22:0/3601751620 shutdown_connections 2023-07-06T01:19:35.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.128+0000 7f22a9b87700 1 -- 172.21.15.22:0/3601751620 wait complete. 2023-07-06T01:19:35.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.129+0000 7f22a9b87700 1 Processor -- start 2023-07-06T01:19:35.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.129+0000 7f22a9b87700 1 -- start start 2023-07-06T01:19:35.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a9b87700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22a406d4a0 0x7f22a40a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a9b87700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22a4070a30 0x7f22a4158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a9b87700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a40cfbf0 0x7f22a40c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a9b87700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a41588c0 con 0x7f22a40cfbf0 2023-07-06T01:19:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a9b87700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40a8af0 con 0x7f22a4070a30 2023-07-06T01:19:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a9b87700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40a8dd0 con 0x7f22a406d4a0 2023-07-06T01:19:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a40cfbf0 0x7f22a40c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a40cfbf0 0x7f22a40c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45804/0 (socket says 172.21.15.22:45804) 2023-07-06T01:19:35.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a3fff700 1 -- 172.21.15.22:0/2115108916 learned_addr learned my addr 172.21.15.22:0/2115108916 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:35.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a2ffd700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22a4070a30 0x7f22a4158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a37fe700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22a406d4a0 0x7f22a40a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a3fff700 1 -- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22a406d4a0 msgr2=0x7f22a40a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a3fff700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22a406d4a0 0x7f22a40a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a3fff700 1 -- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22a4070a30 msgr2=0x7f22a4158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a3fff700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22a4070a30 0x7f22a4158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a3fff700 1 -- 172.21.15.22:0/2115108916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22940094a0 con 0x7f22a40cfbf0 2023-07-06T01:19:35.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.130+0000 7f22a3fff700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a40cfbf0 0x7f22a40c8f40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f229c002230 tx=0x7f229c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:35.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.131+0000 7f229bfff700 1 -- 172.21.15.22:0/2115108916 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c00aac0 con 0x7f22a40cfbf0 2023-07-06T01:19:35.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.131+0000 7f229bfff700 1 -- 172.21.15.22:0/2115108916 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c00ac40 con 0x7f22a40cfbf0 2023-07-06T01:19:35.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.131+0000 7f22a9b87700 1 -- 172.21.15.22:0/2115108916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c0094a0 con 0x7f22a40cfbf0 2023-07-06T01:19:35.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.131+0000 7f229bfff700 1 -- 172.21.15.22:0/2115108916 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c0070d0 con 0x7f22a40cfbf0 2023-07-06T01:19:35.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.131+0000 7f22a9b87700 1 -- 172.21.15.22:0/2115108916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a40a9390 con 0x7f22a40cfbf0 2023-07-06T01:19:35.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.132+0000 7f22a9b87700 1 -- 172.21.15.22:0/2115108916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a40744b0 con 0x7f22a40cfbf0 2023-07-06T01:19:35.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.132+0000 7f229bfff700 1 -- 172.21.15.22:0/2115108916 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f229c007250 con 0x7f22a40cfbf0 2023-07-06T01:19:35.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.132+0000 7f229bfff700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f228406ace0 0x7f228406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.133+0000 7f229bfff700 1 -- 172.21.15.22:0/2115108916 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f229c088f60 con 0x7f22a40cfbf0 2023-07-06T01:19:35.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.133+0000 7f22a37fe700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f228406ace0 0x7f228406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.133+0000 7f22a37fe700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f228406ace0 0x7f228406d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2294009cd0 tx=0x7f2294009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:35.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.137+0000 7f229bfff700 1 -- 172.21.15.22:0/2115108916 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f229c053f90 con 0x7f22a40cfbf0 2023-07-06T01:19:35.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.377+0000 7f22a9b87700 1 -- 172.21.15.22:0/2115108916 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f22a40b21e0 con 0x7f22a40cfbf0 2023-07-06T01:19:35.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.377+0000 7f229bfff700 1 -- 172.21.15.22:0/2115108916 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f229c0575b0 con 0x7f22a40cfbf0 2023-07-06T01:19:35.380 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:35.380 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T01:19:35.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.379+0000 7f22a9b87700 1 -- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f228406ace0 msgr2=0x7f228406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.379+0000 7f22a9b87700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f228406ace0 0x7f228406d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2294009cd0 tx=0x7f2294009210 comp rx=0 tx=0).stop 2023-07-06T01:19:35.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.379+0000 7f22a9b87700 1 -- 172.21.15.22:0/2115108916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a40cfbf0 msgr2=0x7f22a40c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.379+0000 7f22a9b87700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a40cfbf0 0x7f22a40c8f40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f229c002230 tx=0x7f229c00cf50 comp rx=0 tx=0).stop 2023-07-06T01:19:35.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.379+0000 7f22a9b87700 1 -- 172.21.15.22:0/2115108916 shutdown_connections 2023-07-06T01:19:35.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.379+0000 7f22a9b87700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22a40cfbf0 0x7f22a40c8f40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.379+0000 7f22a9b87700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f22a406d4a0 0x7f22a40a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.379+0000 7f22a9b87700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f228406ace0 0x7f228406d190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.379+0000 7f22a9b87700 1 --2- 172.21.15.22:0/2115108916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22a4070a30 0x7f22a4158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.379+0000 7f22a9b87700 1 -- 172.21.15.22:0/2115108916 >> 172.21.15.22:0/2115108916 conn(0x7f22a40b17b0 msgr2=0x7f22a41591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:35.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.379+0000 7f22a9b87700 1 -- 172.21.15.22:0/2115108916 shutdown_connections 2023-07-06T01:19:35.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.379+0000 7f22a9b87700 1 -- 172.21.15.22:0/2115108916 wait complete. 2023-07-06T01:19:35.394 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-07-06T01:19:35.395 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:35.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.517+0000 7f11f6679700 1 Processor -- start 2023-07-06T01:19:35.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.517+0000 7f11f6679700 1 -- start start 2023-07-06T01:19:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.518+0000 7f11f6679700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f11f0158bb0 0x7f11f006b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.518+0000 7f11f6679700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f11f006c900 0x7f11f0070910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.518+0000 7f11f6679700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f0158f80 con 0x7f11f015c6b0 2023-07-06T01:19:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.518+0000 7f11f6679700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f00701e0 con 0x7f11f0158bb0 2023-07-06T01:19:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.518+0000 7f11f6679700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f015ce90 con 0x7f11f006c900 2023-07-06T01:19:35.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.518+0000 7f11f4c16700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f11f006c900 0x7f11f0070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.518+0000 7f11effff700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f11f0158bb0 0x7f11f006b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11effff700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f11f0158bb0 0x7f11f006b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:34532/0 (socket says 172.21.15.22:34532) 2023-07-06T01:19:35.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11f4c16700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f11f006c900 0x7f11f0070910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:56948/0 (socket says 172.21.15.22:56948) 2023-07-06T01:19:35.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11ef7fe700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f11f015c6b0 0x7f11f006c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38620/0 (socket says 172.21.15.22:38620) 2023-07-06T01:19:35.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11effff700 1 -- 172.21.15.22:0/2741825745 learned_addr learned my addr 172.21.15.22:0/2741825745 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:35.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11effff700 1 -- 172.21.15.22:0/2741825745 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f11f006c900 msgr2=0x7f11f0070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:35.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11effff700 1 --2- 172.21.15.22:0/2741825745 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f11f006c900 0x7f11f0070910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11effff700 1 -- 172.21.15.22:0/2741825745 >> v1:172.21.15.103:6789/0 conn(0x7f11f015c6b0 legacy=0x7f11f006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11effff700 1 -- 172.21.15.22:0/2741825745 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f015d170 con 0x7f11f0158bb0 2023-07-06T01:19:35.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11eeffd700 1 -- 172.21.15.22:0/2741825745 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043242991 0 0) 0x7f11f0158f80 con 0x7f11f015c6b0 2023-07-06T01:19:35.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11f4c16700 1 --2- 172.21.15.22:0/2741825745 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f11f006c900 0x7f11f0070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:35.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11effff700 1 --2- 172.21.15.22:0/2741825745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11f0158bb0 0x7f11f006b5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f11e000bb90 tx=0x7f11e000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2752a280a736bce server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:35.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11eeffd700 1 -- 172.21.15.22:0/2741825745 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e000dc20 con 0x7f11f0158bb0 2023-07-06T01:19:35.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11eeffd700 1 -- 172.21.15.22:0/2741825745 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e000dd80 con 0x7f11f0158bb0 2023-07-06T01:19:35.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.519+0000 7f11eeffd700 1 -- 172.21.15.22:0/2741825745 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0017630 con 0x7f11f0158bb0 2023-07-06T01:19:35.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.520+0000 7f11f6679700 1 -- 172.21.15.22:0/2741825745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11f0158bb0 msgr2=0x7f11f006b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.520+0000 7f11f6679700 1 --2- 172.21.15.22:0/2741825745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11f0158bb0 0x7f11f006b5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f11e000bb90 tx=0x7f11e000bea0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.520+0000 7f11f6679700 1 -- 172.21.15.22:0/2741825745 shutdown_connections 2023-07-06T01:19:35.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.520+0000 7f11f6679700 1 --2- 172.21.15.22:0/2741825745 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f11f006c900 0x7f11f0070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.520+0000 7f11f6679700 1 --2- 172.21.15.22:0/2741825745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11f0158bb0 0x7f11f006b5a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.520+0000 7f11f6679700 1 -- 172.21.15.22:0/2741825745 >> 172.21.15.22:0/2741825745 conn(0x7f11f00ccb00 msgr2=0x7f11f00cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:35.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.520+0000 7f11f6679700 1 -- 172.21.15.22:0/2741825745 shutdown_connections 2023-07-06T01:19:35.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.520+0000 7f11f6679700 1 -- 172.21.15.22:0/2741825745 wait complete. 2023-07-06T01:19:35.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.521+0000 7f11f6679700 1 Processor -- start 2023-07-06T01:19:35.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.521+0000 7f11f6679700 1 -- start start 2023-07-06T01:19:35.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11f6679700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11f006c900 0x7f11f014f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11f6679700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f11f0158bb0 0x7f11f00b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11f6679700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11f015c6b0 0x7f11f00c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11f6679700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f014fc10 con 0x7f11f006c900 2023-07-06T01:19:35.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11f6679700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f00c3e40 con 0x7f11f015c6b0 2023-07-06T01:19:35.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11f6679700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f00c4120 con 0x7f11f0158bb0 2023-07-06T01:19:35.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11effff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11f006c900 0x7f11f014f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11effff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11f006c900 0x7f11f014f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45814/0 (socket says 172.21.15.22:45814) 2023-07-06T01:19:35.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11effff700 1 -- 172.21.15.22:0/256156538 learned_addr learned my addr 172.21.15.22:0/256156538 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:35.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11ef7fe700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f11f0158bb0 0x7f11f00b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11effff700 1 -- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f11f0158bb0 msgr2=0x7f11f00b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11f4c16700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11f015c6b0 0x7f11f00c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11effff700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f11f0158bb0 0x7f11f00b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11effff700 1 -- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11f015c6b0 msgr2=0x7f11f00c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11effff700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11f015c6b0 0x7f11f00c4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.522+0000 7f11effff700 1 -- 172.21.15.22:0/256156538 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e8009a70 con 0x7f11f006c900 2023-07-06T01:19:35.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.523+0000 7f11effff700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11f006c900 0x7f11f014f6d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f11e0019040 tx=0x7f11e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:35.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.523+0000 7f11ecff9700 1 -- 172.21.15.22:0/256156538 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e00179e0 con 0x7f11f006c900 2023-07-06T01:19:35.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.523+0000 7f11ecff9700 1 -- 172.21.15.22:0/256156538 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e0017b60 con 0x7f11f006c900 2023-07-06T01:19:35.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.523+0000 7f11f6679700 1 -- 172.21.15.22:0/256156538 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e000b7f0 con 0x7f11f006c900 2023-07-06T01:19:35.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.523+0000 7f11ecff9700 1 -- 172.21.15.22:0/256156538 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e00036a0 con 0x7f11f006c900 2023-07-06T01:19:35.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.523+0000 7f11f6679700 1 -- 172.21.15.22:0/256156538 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f00c46e0 con 0x7f11f006c900 2023-07-06T01:19:35.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.524+0000 7f11f6679700 1 -- 172.21.15.22:0/256156538 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f00744b0 con 0x7f11f006c900 2023-07-06T01:19:35.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.524+0000 7f11ecff9700 1 -- 172.21.15.22:0/256156538 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f11e0015070 con 0x7f11f006c900 2023-07-06T01:19:35.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.524+0000 7f11ecff9700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f11d006ace0 0x7f11d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.525+0000 7f11ecff9700 1 -- 172.21.15.22:0/256156538 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f11e008d3c0 con 0x7f11f006c900 2023-07-06T01:19:35.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.525+0000 7f11ef7fe700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f11d006ace0 0x7f11d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.525+0000 7f11ef7fe700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f11d006ace0 0x7f11d006d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f11f00ce520 tx=0x7f11d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:35.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.529+0000 7f11ecff9700 1 -- 172.21.15.22:0/256156538 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f11e0058480 con 0x7f11f006c900 2023-07-06T01:19:35.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.727+0000 7f11f6679700 1 -- 172.21.15.22:0/256156538 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11f00cd730 con 0x7f11f006c900 2023-07-06T01:19:35.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.728+0000 7f11ecff9700 1 -- 172.21.15.22:0/256156538 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7f11e005baa0 con 0x7f11f006c900 2023-07-06T01:19:35.731 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:35.731 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:35.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.730+0000 7f11f6679700 1 -- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f11d006ace0 msgr2=0x7f11d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.730+0000 7f11f6679700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f11d006ace0 0x7f11d006d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f11f00ce520 tx=0x7f11d8009040 comp rx=0 tx=0).stop 2023-07-06T01:19:35.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.730+0000 7f11f6679700 1 -- 172.21.15.22:0/256156538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11f006c900 msgr2=0x7f11f014f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.730+0000 7f11f6679700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11f006c900 0x7f11f014f6d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f11e0019040 tx=0x7f11e0004750 comp rx=0 tx=0).stop 2023-07-06T01:19:35.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.730+0000 7f11f6679700 1 -- 172.21.15.22:0/256156538 shutdown_connections 2023-07-06T01:19:35.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.730+0000 7f11f6679700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f11d006ace0 0x7f11d006d190 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.730+0000 7f11f6679700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11f015c6b0 0x7f11f00c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.730+0000 7f11f6679700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11f006c900 0x7f11f014f6d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.730+0000 7f11f6679700 1 --2- 172.21.15.22:0/256156538 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f11f0158bb0 0x7f11f00b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.730+0000 7f11f6679700 1 -- 172.21.15.22:0/256156538 >> 172.21.15.22:0/256156538 conn(0x7f11f00ccb00 msgr2=0x7f11f00cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:35.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.730+0000 7f11f6679700 1 -- 172.21.15.22:0/256156538 shutdown_connections 2023-07-06T01:19:35.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.730+0000 7f11f6679700 1 -- 172.21.15.22:0/256156538 wait complete. 2023-07-06T01:19:35.745 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T01:19:22.873255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T01:19:24.019088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:19:35.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:19:35.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.871+0000 7fefcbfb6700 1 Processor -- start 2023-07-06T01:19:35.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.872+0000 7fefcbfb6700 1 -- start start 2023-07-06T01:19:35.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefcbfb6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fefc4159360 0x7fefc415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefcbfb6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefc40ccae0 0x7fefc4071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefcbfb6700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc40cf230 con 0x7fefc40cc660 2023-07-06T01:19:35.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefcbfb6700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc415df20 con 0x7fefc4159360 2023-07-06T01:19:35.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefcbfb6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc40ce620 con 0x7fefc40ccae0 2023-07-06T01:19:35.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefc9d52700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fefc4159360 0x7fefc415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefc9d52700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fefc4159360 0x7fefc415d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45822/0 (socket says 172.21.15.22:45822) 2023-07-06T01:19:35.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefc9d52700 1 -- 172.21.15.22:0/1123634759 learned_addr learned my addr 172.21.15.22:0/1123634759 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:35.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefc9d52700 1 -- 172.21.15.22:0/1123634759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefc40ccae0 msgr2=0x7fefc4071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:35.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefc9d52700 1 --2- 172.21.15.22:0/1123634759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefc40ccae0 0x7fefc4071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefc9d52700 1 -- 172.21.15.22:0/1123634759 >> v1:172.21.15.103:6789/0 conn(0x7fefc40cc660 legacy=0x7fefc4070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefc9d52700 1 -- 172.21.15.22:0/1123634759 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc40ce900 con 0x7fefc4159360 2023-07-06T01:19:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefc9d52700 1 --2- 172.21.15.22:0/1123634759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc4159360 0x7fefc415d9e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fefc000bb90 tx=0x7fefc000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8999efa88c24d8e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:35.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefc8d50700 1 -- 172.21.15.22:0/1123634759 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc000dd10 con 0x7fefc4159360 2023-07-06T01:19:35.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefc8d50700 1 -- 172.21.15.22:0/1123634759 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc000de70 con 0x7fefc4159360 2023-07-06T01:19:35.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.873+0000 7fefc8d50700 1 -- 172.21.15.22:0/1123634759 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0004230 con 0x7fefc4159360 2023-07-06T01:19:35.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.874+0000 7fefcbfb6700 1 -- 172.21.15.22:0/1123634759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc4159360 msgr2=0x7fefc415d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.874+0000 7fefcbfb6700 1 --2- 172.21.15.22:0/1123634759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc4159360 0x7fefc415d9e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fefc000bb90 tx=0x7fefc000bea0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.874+0000 7fefcbfb6700 1 -- 172.21.15.22:0/1123634759 shutdown_connections 2023-07-06T01:19:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.874+0000 7fefcbfb6700 1 --2- 172.21.15.22:0/1123634759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefc40ccae0 0x7fefc4071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.874+0000 7fefcbfb6700 1 --2- 172.21.15.22:0/1123634759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc4159360 0x7fefc415d9e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.874+0000 7fefcbfb6700 1 -- 172.21.15.22:0/1123634759 >> 172.21.15.22:0/1123634759 conn(0x7fefc40b1890 msgr2=0x7fefc40b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.874+0000 7fefcbfb6700 1 -- 172.21.15.22:0/1123634759 shutdown_connections 2023-07-06T01:19:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.874+0000 7fefcbfb6700 1 -- 172.21.15.22:0/1123634759 wait complete. 2023-07-06T01:19:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.875+0000 7fefcbfb6700 1 Processor -- start 2023-07-06T01:19:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.875+0000 7fefcbfb6700 1 -- start start 2023-07-06T01:19:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefcbfb6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefc40cc660 0x7fefc40b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefcbfb6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc40ccae0 0x7fefc406e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefcbfb6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc4159360 0x7fefc40b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefcbfb6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc40acc90 con 0x7fefc4159360 2023-07-06T01:19:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefcbfb6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc40acf50 con 0x7fefc40ccae0 2023-07-06T01:19:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefcbfb6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc40ad230 con 0x7fefc40cc660 2023-07-06T01:19:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefca553700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc4159360 0x7fefc40b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefca553700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc4159360 0x7fefc40b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45832/0 (socket says 172.21.15.22:45832) 2023-07-06T01:19:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefca553700 1 -- 172.21.15.22:0/747599598 learned_addr learned my addr 172.21.15.22:0/747599598 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefc9551700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc40ccae0 0x7fefc406e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefc9d52700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefc40cc660 0x7fefc40b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefca553700 1 -- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefc40cc660 msgr2=0x7fefc40b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefca553700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefc40cc660 0x7fefc40b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefca553700 1 -- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc40ccae0 msgr2=0x7fefc406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefca553700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc40ccae0 0x7fefc406e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefca553700 1 -- 172.21.15.22:0/747599598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc000b7f0 con 0x7fefc4159360 2023-07-06T01:19:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.876+0000 7fefc9551700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc40ccae0 0x7fefc406e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.877+0000 7fefca553700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc4159360 0x7fefc40b2b90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00b970 tx=0x7fefbc00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.877+0000 7fefb67fc700 1 -- 172.21.15.22:0/747599598 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc00c9b0 con 0x7fefc4159360 2023-07-06T01:19:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.877+0000 7fefb67fc700 1 -- 172.21.15.22:0/747599598 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefbc00cb30 con 0x7fefc4159360 2023-07-06T01:19:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.877+0000 7fefb67fc700 1 -- 172.21.15.22:0/747599598 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc014440 con 0x7fefc4159360 2023-07-06T01:19:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.877+0000 7fefcbfb6700 1 -- 172.21.15.22:0/747599598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc40ad510 con 0x7fefc4159360 2023-07-06T01:19:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.877+0000 7fefcbfb6700 1 -- 172.21.15.22:0/747599598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc40adad0 con 0x7fefc4159360 2023-07-06T01:19:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.878+0000 7fefcbfb6700 1 -- 172.21.15.22:0/747599598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc40cf6a0 con 0x7fefc4159360 2023-07-06T01:19:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.878+0000 7fefb67fc700 1 -- 172.21.15.22:0/747599598 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fefbc01d3b0 con 0x7fefc4159360 2023-07-06T01:19:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.879+0000 7fefb67fc700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fefa806ad10 0x7fefa806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.879+0000 7fefb67fc700 1 -- 172.21.15.22:0/747599598 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fefbc006e40 con 0x7fefc4159360 2023-07-06T01:19:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.879+0000 7fefc9d52700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fefa806ad10 0x7fefa806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:35.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.879+0000 7fefc9d52700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fefa806ad10 0x7fefa806d1c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fefc000b340 tx=0x7fefc0013000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:35.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:35.883+0000 7fefb67fc700 1 -- 172.21.15.22:0/747599598 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fefc40cf6a0 con 0x7fefc4159360 2023-07-06T01:19:36.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.078+0000 7fefcbfb6700 1 -- 172.21.15.22:0/747599598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fefc40cf6a0 con 0x7fefc4159360 2023-07-06T01:19:36.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.079+0000 7fefb67fc700 1 -- 172.21.15.22:0/747599598 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fefc40cf6a0 con 0x7fefc4159360 2023-07-06T01:19:36.082 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:19:36.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.081+0000 7fefcbfb6700 1 -- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fefa806ad10 msgr2=0x7fefa806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.081+0000 7fefcbfb6700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fefa806ad10 0x7fefa806d1c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fefc000b340 tx=0x7fefc0013000 comp rx=0 tx=0).stop 2023-07-06T01:19:36.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.081+0000 7fefcbfb6700 1 -- 172.21.15.22:0/747599598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc4159360 msgr2=0x7fefc40b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.081+0000 7fefcbfb6700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc4159360 0x7fefc40b2b90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00b970 tx=0x7fefbc00bd30 comp rx=0 tx=0).stop 2023-07-06T01:19:36.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.081+0000 7fefcbfb6700 1 -- 172.21.15.22:0/747599598 shutdown_connections 2023-07-06T01:19:36.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.081+0000 7fefcbfb6700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefc4159360 0x7fefc40b2b90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.081+0000 7fefcbfb6700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefc40cc660 0x7fefc40b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.081+0000 7fefcbfb6700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fefa806ad10 0x7fefa806d1c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.081+0000 7fefcbfb6700 1 --2- 172.21.15.22:0/747599598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefc40ccae0 0x7fefc406e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.081+0000 7fefcbfb6700 1 -- 172.21.15.22:0/747599598 >> 172.21.15.22:0/747599598 conn(0x7fefc40b1890 msgr2=0x7fefc4155cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:36.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.081+0000 7fefcbfb6700 1 -- 172.21.15.22:0/747599598 shutdown_connections 2023-07-06T01:19:36.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.081+0000 7fefcbfb6700 1 -- 172.21.15.22:0/747599598 wait complete. 2023-07-06T01:19:36.096 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:19:36.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.222+0000 7f2499dea700 1 Processor -- start 2023-07-06T01:19:36.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.222+0000 7f2499dea700 1 -- start start 2023-07-06T01:19:36.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.223+0000 7f2499dea700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f24940af2a0 0x7f249406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.223+0000 7f2499dea700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f249406a000 0x7f249406a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.223+0000 7f2499dea700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24940af670 con 0x7f24940cbc10 2023-07-06T01:19:36.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.223+0000 7f2499dea700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249406a9e0 con 0x7f24940af2a0 2023-07-06T01:19:36.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.223+0000 7f2499dea700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24940cc7a0 con 0x7f249406a000 2023-07-06T01:19:36.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.223+0000 7f24937fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f24940af2a0 0x7f249406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.223+0000 7f2493fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f24940cbc10 0x7f24940cc090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36584/0 (socket says 172.21.15.22:36584) 2023-07-06T01:19:36.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.223+0000 7f24937fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f24940af2a0 0x7f249406caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45836/0 (socket says 172.21.15.22:45836) 2023-07-06T01:19:36.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.223+0000 7f2493fff700 1 -- 172.21.15.22:0/4056020586 learned_addr learned my addr 172.21.15.22:0/4056020586 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:36.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.223+0000 7f2492ffd700 1 --2- 172.21.15.22:0/4056020586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f249406a000 0x7f249406a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.224+0000 7f24937fe700 1 -- 172.21.15.22:0/4056020586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f249406a000 msgr2=0x7f249406a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:36.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.224+0000 7f24937fe700 1 --2- 172.21.15.22:0/4056020586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f249406a000 0x7f249406a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.224+0000 7f24927fc700 1 -- 172.21.15.22:0/4056020586 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385907006 0 0) 0x7f24940af670 con 0x7f24940cbc10 2023-07-06T01:19:36.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.224+0000 7f24937fe700 1 -- 172.21.15.22:0/4056020586 >> v1:172.21.15.22:6789/0 conn(0x7f24940cbc10 legacy=0x7f24940cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.224+0000 7f24937fe700 1 -- 172.21.15.22:0/4056020586 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249406d030 con 0x7f24940af2a0 2023-07-06T01:19:36.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.224+0000 7f24937fe700 1 --2- 172.21.15.22:0/4056020586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24940af2a0 0x7f249406caf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2484009840 tx=0x7f2484009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343efb1ad4b41fb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:36.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.224+0000 7f24927fc700 1 -- 172.21.15.22:0/4056020586 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484011830 con 0x7f24940af2a0 2023-07-06T01:19:36.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.224+0000 7f24927fc700 1 -- 172.21.15.22:0/4056020586 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2484011990 con 0x7f24940af2a0 2023-07-06T01:19:36.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.224+0000 7f24927fc700 1 -- 172.21.15.22:0/4056020586 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484011d20 con 0x7f24940af2a0 2023-07-06T01:19:36.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.224+0000 7f2499dea700 1 -- 172.21.15.22:0/4056020586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24940af2a0 msgr2=0x7f249406caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.224+0000 7f2499dea700 1 --2- 172.21.15.22:0/4056020586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24940af2a0 0x7f249406caf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2484009840 tx=0x7f2484009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:36.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.225+0000 7f2499dea700 1 -- 172.21.15.22:0/4056020586 shutdown_connections 2023-07-06T01:19:36.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.225+0000 7f2499dea700 1 --2- 172.21.15.22:0/4056020586 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24940af2a0 0x7f249406caf0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.225+0000 7f2499dea700 1 --2- 172.21.15.22:0/4056020586 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f249406a000 0x7f249406a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.225+0000 7f2499dea700 1 -- 172.21.15.22:0/4056020586 >> 172.21.15.22:0/4056020586 conn(0x7f24940b1890 msgr2=0x7f24940b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:36.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.225+0000 7f2499dea700 1 -- 172.21.15.22:0/4056020586 shutdown_connections 2023-07-06T01:19:36.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.225+0000 7f2499dea700 1 -- 172.21.15.22:0/4056020586 wait complete. 2023-07-06T01:19:36.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.226+0000 7f2499dea700 1 Processor -- start 2023-07-06T01:19:36.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.226+0000 7f2499dea700 1 -- start start 2023-07-06T01:19:36.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2499dea700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f249406a000 0x7f2494159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2499dea700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24940af2a0 0x7f249415a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2499dea700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24940cbc10 0x7f249415aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2499dea700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24940b30e0 con 0x7f24940cbc10 2023-07-06T01:19:36.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2499dea700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249415a360 con 0x7f249406a000 2023-07-06T01:19:36.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2499dea700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249415ce40 con 0x7f24940af2a0 2023-07-06T01:19:36.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2493fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24940cbc10 0x7f249415aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2493fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24940cbc10 0x7f249415aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45846/0 (socket says 172.21.15.22:45846) 2023-07-06T01:19:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2493fff700 1 -- 172.21.15.22:0/960787440 learned_addr learned my addr 172.21.15.22:0/960787440 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2492ffd700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24940af2a0 0x7f249415a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f24937fe700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f249406a000 0x7f2494159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2493fff700 1 -- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24940af2a0 msgr2=0x7f249415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2493fff700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24940af2a0 0x7f249415a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2493fff700 1 -- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f249406a000 msgr2=0x7f2494159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2493fff700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f249406a000 0x7f2494159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2493fff700 1 -- 172.21.15.22:0/960787440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24840094a0 con 0x7f24940cbc10 2023-07-06T01:19:36.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f2493fff700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24940cbc10 0x7f249415aaf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f248c00cba0 tx=0x7f248c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:36.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.227+0000 7f248bfff700 1 -- 172.21.15.22:0/960787440 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c00aaa0 con 0x7f24940cbc10 2023-07-06T01:19:36.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.228+0000 7f248bfff700 1 -- 172.21.15.22:0/960787440 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248c00ac20 con 0x7f24940cbc10 2023-07-06T01:19:36.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.228+0000 7f2499dea700 1 -- 172.21.15.22:0/960787440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249415d120 con 0x7f24940cbc10 2023-07-06T01:19:36.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.228+0000 7f248bfff700 1 -- 172.21.15.22:0/960787440 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c0070d0 con 0x7f24940cbc10 2023-07-06T01:19:36.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.228+0000 7f2499dea700 1 -- 172.21.15.22:0/960787440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249415d6e0 con 0x7f24940cbc10 2023-07-06T01:19:36.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.229+0000 7f2499dea700 1 -- 172.21.15.22:0/960787440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249415d900 con 0x7f24940cbc10 2023-07-06T01:19:36.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.229+0000 7f248bfff700 1 -- 172.21.15.22:0/960787440 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f248c007230 con 0x7f24940cbc10 2023-07-06T01:19:36.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.229+0000 7f248bfff700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f247406ace0 0x7f247406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.229+0000 7f248bfff700 1 -- 172.21.15.22:0/960787440 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f248c0891a0 con 0x7f24940cbc10 2023-07-06T01:19:36.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.230+0000 7f24937fe700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f247406ace0 0x7f247406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.230+0000 7f24937fe700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f247406ace0 0x7f247406d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f24840090b0 tx=0x7f2484008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:36.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.234+0000 7f248bfff700 1 -- 172.21.15.22:0/960787440 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f248c054240 con 0x7f24940cbc10 2023-07-06T01:19:36.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.428+0000 7f2499dea700 1 -- 172.21.15.22:0/960787440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f24940744b0 con 0x7f24940cbc10 2023-07-06T01:19:36.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.428+0000 7f248bfff700 1 -- 172.21.15.22:0/960787440 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f248c057860 con 0x7f24940cbc10 2023-07-06T01:19:36.431 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:19:36.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.431+0000 7f2499dea700 1 -- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f247406ace0 msgr2=0x7f247406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.431+0000 7f2499dea700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f247406ace0 0x7f247406d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f24840090b0 tx=0x7f2484008030 comp rx=0 tx=0).stop 2023-07-06T01:19:36.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.431+0000 7f2499dea700 1 -- 172.21.15.22:0/960787440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24940cbc10 msgr2=0x7f249415aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.431+0000 7f2499dea700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24940cbc10 0x7f249415aaf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f248c00cba0 tx=0x7f248c00ceb0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.431+0000 7f2499dea700 1 -- 172.21.15.22:0/960787440 shutdown_connections 2023-07-06T01:19:36.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.431+0000 7f2499dea700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24940cbc10 0x7f249415aaf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.431+0000 7f2499dea700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f247406ace0 0x7f247406d190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.431+0000 7f2499dea700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f249406a000 0x7f2494159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.431+0000 7f2499dea700 1 --2- 172.21.15.22:0/960787440 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24940af2a0 0x7f249415a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.431+0000 7f2499dea700 1 -- 172.21.15.22:0/960787440 >> 172.21.15.22:0/960787440 conn(0x7f24940b1890 msgr2=0x7f249406d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:36.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.431+0000 7f2499dea700 1 -- 172.21.15.22:0/960787440 shutdown_connections 2023-07-06T01:19:36.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.431+0000 7f2499dea700 1 -- 172.21.15.22:0/960787440 wait complete. 2023-07-06T01:19:36.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T01:19:36.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.571+0000 7f6582a4e700 1 Processor -- start 2023-07-06T01:19:36.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.571+0000 7f6582a4e700 1 -- start start 2023-07-06T01:19:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.572+0000 7f6582a4e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f657c15c750 0x7f657c0cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.572+0000 7f6582a4e700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f657c0cdf80 0x7f657c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.572+0000 7f6582a4e700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c0d06f0 con 0x7f657c0ce3d0 2023-07-06T01:19:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.572+0000 7f6582a4e700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657c0ceb90 con 0x7f657c0cdf80 2023-07-06T01:19:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.572+0000 7f6582a4e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0cee70 con 0x7f657c15c750 2023-07-06T01:19:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.572+0000 7f657bfff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f657c15c750 0x7f657c0cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.572+0000 7f657b7fe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f657c0cdf80 0x7f657c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.572+0000 7f6580feb700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f657c0ce3d0 0x7f657c0cb2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38662/0 (socket says 172.21.15.22:38662) 2023-07-06T01:19:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.572+0000 7f6580feb700 1 -- 172.21.15.22:0/3612872365 learned_addr learned my addr 172.21.15.22:0/3612872365 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.572+0000 7f657bfff700 1 -- 172.21.15.22:0/3612872365 >> v2:172.21.15.103:3300/0 conn(0x7f657c0cdf80 msgr2=0x7f657c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.572+0000 7f657affd700 1 -- 172.21.15.22:0/3612872365 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213785801 0 0) 0x7f657c0d06f0 con 0x7f657c0ce3d0 2023-07-06T01:19:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.572+0000 7f657bfff700 1 --2- 172.21.15.22:0/3612872365 >> v2:172.21.15.103:3300/0 conn(0x7f657c0cdf80 0x7f657c06b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.572+0000 7f657bfff700 1 -- 172.21.15.22:0/3612872365 >> v1:172.21.15.103:6789/0 conn(0x7f657c0ce3d0 legacy=0x7f657c0cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.573+0000 7f657bfff700 1 -- 172.21.15.22:0/3612872365 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c0cf150 con 0x7f657c15c750 2023-07-06T01:19:36.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.573+0000 7f657bfff700 1 --2- 172.21.15.22:0/3612872365 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f657c15c750 0x7f657c0cfc30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6564009840 tx=0x7f6564009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=addca07827e68a37 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:36.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.573+0000 7f657affd700 1 -- 172.21.15.22:0/3612872365 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564011830 con 0x7f657c15c750 2023-07-06T01:19:36.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.573+0000 7f657affd700 1 -- 172.21.15.22:0/3612872365 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6564011990 con 0x7f657c15c750 2023-07-06T01:19:36.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.573+0000 7f657affd700 1 -- 172.21.15.22:0/3612872365 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564011d20 con 0x7f657c15c750 2023-07-06T01:19:36.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.573+0000 7f6582a4e700 1 -- 172.21.15.22:0/3612872365 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f657c15c750 msgr2=0x7f657c0cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.573+0000 7f6582a4e700 1 --2- 172.21.15.22:0/3612872365 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f657c15c750 0x7f657c0cfc30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6564009840 tx=0x7f6564009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:36.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.573+0000 7f6582a4e700 1 -- 172.21.15.22:0/3612872365 shutdown_connections 2023-07-06T01:19:36.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.573+0000 7f6582a4e700 1 --2- 172.21.15.22:0/3612872365 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f657c15c750 0x7f657c0cfc30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.573+0000 7f6582a4e700 1 --2- 172.21.15.22:0/3612872365 >> v2:172.21.15.103:3300/0 conn(0x7f657c0cdf80 0x7f657c06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.573+0000 7f6582a4e700 1 -- 172.21.15.22:0/3612872365 >> 172.21.15.22:0/3612872365 conn(0x7f657c0999f0 msgr2=0x7f657c099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:36.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.574+0000 7f6582a4e700 1 -- 172.21.15.22:0/3612872365 shutdown_connections 2023-07-06T01:19:36.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.574+0000 7f6582a4e700 1 -- 172.21.15.22:0/3612872365 wait complete. 2023-07-06T01:19:36.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.575+0000 7f6582a4e700 1 Processor -- start 2023-07-06T01:19:36.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.575+0000 7f6582a4e700 1 -- start start 2023-07-06T01:19:36.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6582a4e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f657c0cdf80 0x7f657c1583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6582a4e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f657c0ce3d0 0x7f657c152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6582a4e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f657c15c750 0x7f657c153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6582a4e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c156de0 con 0x7f657c15c750 2023-07-06T01:19:36.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6582a4e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0b0df0 con 0x7f657c0cdf80 2023-07-06T01:19:36.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6582a4e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c158aa0 con 0x7f657c0ce3d0 2023-07-06T01:19:36.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6580feb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f657c15c750 0x7f657c153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6580feb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f657c15c750 0x7f657c153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45860/0 (socket says 172.21.15.22:45860) 2023-07-06T01:19:36.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f657bfff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f657c0cdf80 0x7f657c1583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6580feb700 1 -- 172.21.15.22:0/297513941 learned_addr learned my addr 172.21.15.22:0/297513941 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:36.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f657b7fe700 1 --2- 172.21.15.22:0/297513941 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f657c0ce3d0 0x7f657c152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6580feb700 1 -- 172.21.15.22:0/297513941 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f657c0ce3d0 msgr2=0x7f657c152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6580feb700 1 --2- 172.21.15.22:0/297513941 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f657c0ce3d0 0x7f657c152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6580feb700 1 -- 172.21.15.22:0/297513941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f657c0cdf80 msgr2=0x7f657c1583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6580feb700 1 --2- 172.21.15.22:0/297513941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f657c0cdf80 0x7f657c1583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6580feb700 1 -- 172.21.15.22:0/297513941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65640094a0 con 0x7f657c15c750 2023-07-06T01:19:36.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.576+0000 7f6580feb700 1 --2- 172.21.15.22:0/297513941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f657c15c750 0x7f657c153460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f657400cc70 tx=0x7f657400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:36.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.577+0000 7f6578ff9700 1 -- 172.21.15.22:0/297513941 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657400aaa0 con 0x7f657c15c750 2023-07-06T01:19:36.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.577+0000 7f6582a4e700 1 -- 172.21.15.22:0/297513941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c0b1090 con 0x7f657c15c750 2023-07-06T01:19:36.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.577+0000 7f6578ff9700 1 -- 172.21.15.22:0/297513941 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657400ac20 con 0x7f657c15c750 2023-07-06T01:19:36.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.577+0000 7f6578ff9700 1 -- 172.21.15.22:0/297513941 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65740070d0 con 0x7f657c15c750 2023-07-06T01:19:36.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.577+0000 7f6582a4e700 1 -- 172.21.15.22:0/297513941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c0b1650 con 0x7f657c15c750 2023-07-06T01:19:36.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.578+0000 7f6582a4e700 1 -- 172.21.15.22:0/297513941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c0b1840 con 0x7f657c15c750 2023-07-06T01:19:36.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.578+0000 7f6578ff9700 1 -- 172.21.15.22:0/297513941 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6574007230 con 0x7f657c15c750 2023-07-06T01:19:36.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.578+0000 7f6578ff9700 1 --2- 172.21.15.22:0/297513941 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f655c06ace0 0x7f655c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.578+0000 7f6578ff9700 1 -- 172.21.15.22:0/297513941 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6574088fb0 con 0x7f657c15c750 2023-07-06T01:19:36.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.579+0000 7f657bfff700 1 --2- 172.21.15.22:0/297513941 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f655c06ace0 0x7f655c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.579+0000 7f657bfff700 1 --2- 172.21.15.22:0/297513941 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f655c06ace0 0x7f655c06d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f65640090b0 tx=0x7f6564008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:36.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.583+0000 7f6578ff9700 1 -- 172.21.15.22:0/297513941 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f65740bb310 con 0x7f657c15c750 2023-07-06T01:19:36.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.781+0000 7f6582a4e700 1 -- 172.21.15.22:0/297513941 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f657c0744b0 con 0x7f657c15c750 2023-07-06T01:19:36.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.781+0000 7f6578ff9700 1 -- 172.21.15.22:0/297513941 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6574054070 con 0x7f657c15c750 2023-07-06T01:19:36.784 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-07-06T01:19:36.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.783+0000 7f6582a4e700 1 -- 172.21.15.22:0/297513941 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f655c06ace0 msgr2=0x7f655c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.783+0000 7f6582a4e700 1 --2- 172.21.15.22:0/297513941 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f655c06ace0 0x7f655c06d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f65640090b0 tx=0x7f6564008030 comp rx=0 tx=0).stop 2023-07-06T01:19:36.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.783+0000 7f6582a4e700 1 -- 172.21.15.22:0/297513941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f657c15c750 msgr2=0x7f657c153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.783+0000 7f6582a4e700 1 --2- 172.21.15.22:0/297513941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f657c15c750 0x7f657c153460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f657400cc70 tx=0x7f657400cf80 comp rx=0 tx=0).stop 2023-07-06T01:19:36.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.783+0000 7f6582a4e700 1 -- 172.21.15.22:0/297513941 shutdown_connections 2023-07-06T01:19:36.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.783+0000 7f6582a4e700 1 --2- 172.21.15.22:0/297513941 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f657c15c750 0x7f657c153460 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.783+0000 7f6582a4e700 1 --2- 172.21.15.22:0/297513941 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f655c06ace0 0x7f655c06d190 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.783+0000 7f6582a4e700 1 --2- 172.21.15.22:0/297513941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f657c0cdf80 0x7f657c1583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.783+0000 7f6582a4e700 1 --2- 172.21.15.22:0/297513941 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f657c0ce3d0 0x7f657c152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.783+0000 7f6582a4e700 1 -- 172.21.15.22:0/297513941 >> 172.21.15.22:0/297513941 conn(0x7f657c0999f0 msgr2=0x7f657c06b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:36.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.783+0000 7f6582a4e700 1 -- 172.21.15.22:0/297513941 shutdown_connections 2023-07-06T01:19:36.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.783+0000 7f6582a4e700 1 -- 172.21.15.22:0/297513941 wait complete. 2023-07-06T01:19:36.798 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T01:19:36.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.922+0000 7f1d93cdf700 1 Processor -- start 2023-07-06T01:19:36.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.923+0000 7f1d93cdf700 1 -- start start 2023-07-06T01:19:36.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.923+0000 7f1d93cdf700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1d8c15c940 0x7f1d8c0d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.923+0000 7f1d93cdf700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d8c15cd10 con 0x7f1d8c0cd510 2023-07-06T01:19:36.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.923+0000 7f1d93cdf700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d8c0ce710 con 0x7f1d8c0d11a0 2023-07-06T01:19:36.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.923+0000 7f1d93cdf700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8c09c8a0 con 0x7f1d8c15c940 2023-07-06T01:19:36.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.924+0000 7f1d91a7b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1d8c15c940 0x7f1d8c0d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.924+0000 7f1d9127a700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1d8c0cd510 0x7f1d8c0d0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36608/0 (socket says 172.21.15.22:36608) 2023-07-06T01:19:36.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.924+0000 7f1d9127a700 1 -- 172.21.15.22:0/2875970834 learned_addr learned my addr 172.21.15.22:0/2875970834 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:36.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.924+0000 7f1d91a7b700 1 -- 172.21.15.22:0/2875970834 >> v1:172.21.15.103:6789/0 conn(0x7f1d8c0d11a0 legacy=0x7f1d8c0afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.924+0000 7f1d91a7b700 1 -- 172.21.15.22:0/2875970834 >> v1:172.21.15.22:6789/0 conn(0x7f1d8c0cd510 legacy=0x7f1d8c0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.924+0000 7f1d91a7b700 1 -- 172.21.15.22:0/2875970834 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c0ce9f0 con 0x7f1d8c15c940 2023-07-06T01:19:36.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.924+0000 7f1d91a7b700 1 --2- 172.21.15.22:0/2875970834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d8c15c940 0x7f1d8c0d2a10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009840 tx=0x7f1d7c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed7cecd59355131 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.924+0000 7f1d90a79700 1 -- 172.21.15.22:0/2875970834 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c011830 con 0x7f1d8c15c940 2023-07-06T01:19:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.924+0000 7f1d90a79700 1 -- 172.21.15.22:0/2875970834 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c011990 con 0x7f1d8c15c940 2023-07-06T01:19:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.924+0000 7f1d90a79700 1 -- 172.21.15.22:0/2875970834 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c011d20 con 0x7f1d8c15c940 2023-07-06T01:19:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.925+0000 7f1d93cdf700 1 -- 172.21.15.22:0/2875970834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d8c15c940 msgr2=0x7f1d8c0d2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.925+0000 7f1d93cdf700 1 --2- 172.21.15.22:0/2875970834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d8c15c940 0x7f1d8c0d2a10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009840 tx=0x7f1d7c009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.925+0000 7f1d93cdf700 1 -- 172.21.15.22:0/2875970834 shutdown_connections 2023-07-06T01:19:36.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.925+0000 7f1d93cdf700 1 --2- 172.21.15.22:0/2875970834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d8c15c940 0x7f1d8c0d2a10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.925+0000 7f1d93cdf700 1 -- 172.21.15.22:0/2875970834 >> 172.21.15.22:0/2875970834 conn(0x7f1d8c09c2e0 msgr2=0x7f1d8c0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:36.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.925+0000 7f1d93cdf700 1 -- 172.21.15.22:0/2875970834 shutdown_connections 2023-07-06T01:19:36.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.925+0000 7f1d93cdf700 1 -- 172.21.15.22:0/2875970834 wait complete. 2023-07-06T01:19:36.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.926+0000 7f1d93cdf700 1 Processor -- start 2023-07-06T01:19:36.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.926+0000 7f1d93cdf700 1 -- start start 2023-07-06T01:19:36.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d93cdf700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d8c0cd510 0x7f1d8c14c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d93cdf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d8c0d11a0 0x7f1d8c0cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d93cdf700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d8c15c940 0x7f1d8c0b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d93cdf700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c14c950 con 0x7f1d8c0d11a0 2023-07-06T01:19:36.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d93cdf700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c1e7100 con 0x7f1d8c0cd510 2023-07-06T01:19:36.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d93cdf700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c1e73e0 con 0x7f1d8c15c940 2023-07-06T01:19:36.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d9127a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d8c0d11a0 0x7f1d8c0cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d9127a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d8c0d11a0 0x7f1d8c0cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45872/0 (socket says 172.21.15.22:45872) 2023-07-06T01:19:36.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d9127a700 1 -- 172.21.15.22:0/722477669 learned_addr learned my addr 172.21.15.22:0/722477669 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:36.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d91a7b700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d8c0cd510 0x7f1d8c14c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d9227c700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d8c15c940 0x7f1d8c0b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d9127a700 1 -- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d8c15c940 msgr2=0x7f1d8c0b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d9127a700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d8c15c940 0x7f1d8c0b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d9127a700 1 -- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d8c0cd510 msgr2=0x7f1d8c14c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:36.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d9127a700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d8c0cd510 0x7f1d8c14c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:36.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d9127a700 1 -- 172.21.15.22:0/722477669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c0094a0 con 0x7f1d8c0d11a0 2023-07-06T01:19:36.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.927+0000 7f1d9127a700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d8c0d11a0 0x7f1d8c0cbef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800c520 tx=0x7f1d8800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:36.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.928+0000 7f1d827fc700 1 -- 172.21.15.22:0/722477669 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88005d70 con 0x7f1d8c0d11a0 2023-07-06T01:19:36.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.928+0000 7f1d827fc700 1 -- 172.21.15.22:0/722477669 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d880070d0 con 0x7f1d8c0d11a0 2023-07-06T01:19:36.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.928+0000 7f1d93cdf700 1 -- 172.21.15.22:0/722477669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8c1e76c0 con 0x7f1d8c0d11a0 2023-07-06T01:19:36.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.928+0000 7f1d827fc700 1 -- 172.21.15.22:0/722477669 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88007440 con 0x7f1d8c0d11a0 2023-07-06T01:19:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.928+0000 7f1d93cdf700 1 -- 172.21.15.22:0/722477669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8c1e7c80 con 0x7f1d8c0d11a0 2023-07-06T01:19:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.929+0000 7f1d93cdf700 1 -- 172.21.15.22:0/722477669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d8c1e7e70 con 0x7f1d8c0d11a0 2023-07-06T01:19:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.929+0000 7f1d827fc700 1 -- 172.21.15.22:0/722477669 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d880075a0 con 0x7f1d8c0d11a0 2023-07-06T01:19:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.929+0000 7f1d827fc700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d7006ace0 0x7f1d7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.929+0000 7f1d827fc700 1 -- 172.21.15.22:0/722477669 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1d880888f0 con 0x7f1d8c0d11a0 2023-07-06T01:19:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.930+0000 7f1d91a7b700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d7006ace0 0x7f1d7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:36.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.930+0000 7f1d91a7b700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d7006ace0 0x7f1d7006d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0090b0 tx=0x7f1d7c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:36.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:36.934+0000 7f1d827fc700 1 -- 172.21.15.22:0/722477669 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1d88053a50 con 0x7f1d8c0d11a0 2023-07-06T01:19:37.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.131+0000 7f1d93cdf700 1 -- 172.21.15.22:0/722477669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d8c0744b0 con 0x7f1d8c0d11a0 2023-07-06T01:19:37.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.132+0000 7f1d827fc700 1 -- 172.21.15.22:0/722477669 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d88057070 con 0x7f1d8c0d11a0 2023-07-06T01:19:37.135 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-07-06T01:19:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.134+0000 7f1d93cdf700 1 -- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d7006ace0 msgr2=0x7f1d7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.134+0000 7f1d93cdf700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d7006ace0 0x7f1d7006d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0090b0 tx=0x7f1d7c01c040 comp rx=0 tx=0).stop 2023-07-06T01:19:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.134+0000 7f1d93cdf700 1 -- 172.21.15.22:0/722477669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d8c0d11a0 msgr2=0x7f1d8c0cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.134+0000 7f1d93cdf700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d8c0d11a0 0x7f1d8c0cbef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800c520 tx=0x7f1d8800c8e0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.134+0000 7f1d93cdf700 1 -- 172.21.15.22:0/722477669 shutdown_connections 2023-07-06T01:19:37.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.134+0000 7f1d93cdf700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d8c15c940 0x7f1d8c0b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.134+0000 7f1d93cdf700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d7006ace0 0x7f1d7006d190 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.134+0000 7f1d93cdf700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d8c0cd510 0x7f1d8c14c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.134+0000 7f1d93cdf700 1 --2- 172.21.15.22:0/722477669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d8c0d11a0 0x7f1d8c0cbef0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.134+0000 7f1d93cdf700 1 -- 172.21.15.22:0/722477669 >> 172.21.15.22:0/722477669 conn(0x7f1d8c09c2e0 msgr2=0x7f1d8c0b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:37.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.134+0000 7f1d93cdf700 1 -- 172.21.15.22:0/722477669 shutdown_connections 2023-07-06T01:19:37.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.134+0000 7f1d93cdf700 1 -- 172.21.15.22:0/722477669 wait complete. 2023-07-06T01:19:37.148 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:19:37.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.271+0000 7f4bc6b7a700 1 Processor -- start 2023-07-06T01:19:37.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.271+0000 7f4bc6b7a700 1 -- start start 2023-07-06T01:19:37.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bc6b7a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4bc006a710 0x7f4bc006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:37.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bc6b7a700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f4bc00cbc10 0x7f4bc00cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:37.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bc6b7a700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc00af670 con 0x7f4bc00af2a0 2023-07-06T01:19:37.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bc6b7a700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc00cd560 con 0x7f4bc006a710 2023-07-06T01:19:37.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bc6b7a700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc015cab0 con 0x7f4bc00cbc10 2023-07-06T01:19:37.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bbffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4bc006a710 0x7f4bc006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:37.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bbffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4bc006a710 0x7f4bc006caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45880/0 (socket says 172.21.15.22:45880) 2023-07-06T01:19:37.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bbffff700 1 -- 172.21.15.22:0/2848782597 learned_addr learned my addr 172.21.15.22:0/2848782597 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:37.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bc5117700 1 --2- 172.21.15.22:0/2848782597 >> v2:172.21.15.103:3300/0 conn(0x7f4bc00cbc10 0x7f4bc00cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:37.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bbffff700 1 -- 172.21.15.22:0/2848782597 >> v2:172.21.15.103:3300/0 conn(0x7f4bc00cbc10 msgr2=0x7f4bc00cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:37.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bbffff700 1 --2- 172.21.15.22:0/2848782597 >> v2:172.21.15.103:3300/0 conn(0x7f4bc00cbc10 0x7f4bc00cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bbffff700 1 -- 172.21.15.22:0/2848782597 >> v1:172.21.15.103:6789/0 conn(0x7f4bc00af2a0 legacy=0x7f4bc006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:37.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bbffff700 1 -- 172.21.15.22:0/2848782597 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc006d030 con 0x7f4bc006a710 2023-07-06T01:19:37.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.272+0000 7f4bc5117700 1 --2- 172.21.15.22:0/2848782597 >> v2:172.21.15.103:3300/0 conn(0x7f4bc00cbc10 0x7f4bc00cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:37.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.273+0000 7f4bbffff700 1 --2- 172.21.15.22:0/2848782597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc006a710 0x7f4bc006caf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00ca740 tx=0x7f4bac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb280ca1180e1ef2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:37.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.273+0000 7f4bbf7fe700 1 -- 172.21.15.22:0/2848782597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac00b870 con 0x7f4bc006a710 2023-07-06T01:19:37.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.273+0000 7f4bbf7fe700 1 -- 172.21.15.22:0/2848782597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bac00b9d0 con 0x7f4bc006a710 2023-07-06T01:19:37.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.273+0000 7f4bbf7fe700 1 -- 172.21.15.22:0/2848782597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac00bd60 con 0x7f4bc006a710 2023-07-06T01:19:37.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.273+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/2848782597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc006a710 msgr2=0x7f4bc006caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.273+0000 7f4bc6b7a700 1 --2- 172.21.15.22:0/2848782597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc006a710 0x7f4bc006caf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00ca740 tx=0x7f4bac009bd0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.273+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/2848782597 shutdown_connections 2023-07-06T01:19:37.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.273+0000 7f4bc6b7a700 1 --2- 172.21.15.22:0/2848782597 >> v2:172.21.15.103:3300/0 conn(0x7f4bc00cbc10 0x7f4bc00cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.273+0000 7f4bc6b7a700 1 --2- 172.21.15.22:0/2848782597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc006a710 0x7f4bc006caf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.279 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.273+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/2848782597 >> 172.21.15.22:0/2848782597 conn(0x7f4bc00b1890 msgr2=0x7f4bc00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:37.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.273+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/2848782597 shutdown_connections 2023-07-06T01:19:37.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.273+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/2848782597 wait complete. 2023-07-06T01:19:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.274+0000 7f4bc6b7a700 1 Processor -- start 2023-07-06T01:19:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.275+0000 7f4bc6b7a700 1 -- start start 2023-07-06T01:19:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.275+0000 7f4bc6b7a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00af2a0 0x7f4bc015e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.275+0000 7f4bc6b7a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bc00cbc10 0x7f4bc014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:37.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.275+0000 7f4bc6b7a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bc01561d0 0x7f4bc00b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:37.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.275+0000 7f4bc6b7a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc015de30 con 0x7f4bc00af2a0 2023-07-06T01:19:37.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.275+0000 7f4bc6b7a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00c94d0 con 0x7f4bc01561d0 2023-07-06T01:19:37.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.275+0000 7f4bc6b7a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00c97b0 con 0x7f4bc00cbc10 2023-07-06T01:19:37.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.275+0000 7f4bc4916700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00af2a0 0x7f4bc015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:37.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.275+0000 7f4bc4916700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00af2a0 0x7f4bc015e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45884/0 (socket says 172.21.15.22:45884) 2023-07-06T01:19:37.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.275+0000 7f4bc4916700 1 -- 172.21.15.22:0/1274001721 learned_addr learned my addr 172.21.15.22:0/1274001721 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:37.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.276+0000 7f4bbffff700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bc00cbc10 0x7f4bc014cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:37.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.276+0000 7f4bc5117700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bc01561d0 0x7f4bc00b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:37.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.276+0000 7f4bc4916700 1 -- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bc00cbc10 msgr2=0x7f4bc014cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.276+0000 7f4bc4916700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bc00cbc10 0x7f4bc014cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.276+0000 7f4bc4916700 1 -- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bc01561d0 msgr2=0x7f4bc00b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.276+0000 7f4bc4916700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bc01561d0 0x7f4bc00b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.276+0000 7f4bc4916700 1 -- 172.21.15.22:0/1274001721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac009550 con 0x7f4bc00af2a0 2023-07-06T01:19:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.276+0000 7f4bc4916700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00af2a0 0x7f4bc015e250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400c4e0 tx=0x7f4bb400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.276+0000 7f4bbd7fa700 1 -- 172.21.15.22:0/1274001721 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40052d0 con 0x7f4bc00af2a0 2023-07-06T01:19:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.276+0000 7f4bbd7fa700 1 -- 172.21.15.22:0/1274001721 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb4005430 con 0x7f4bc00af2a0 2023-07-06T01:19:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.276+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/1274001721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc015d770 con 0x7f4bc00af2a0 2023-07-06T01:19:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.276+0000 7f4bbd7fa700 1 -- 172.21.15.22:0/1274001721 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40038c0 con 0x7f4bc00af2a0 2023-07-06T01:19:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.276+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/1274001721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc014d420 con 0x7f4bc00af2a0 2023-07-06T01:19:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.277+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/1274001721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc00744b0 con 0x7f4bc00af2a0 2023-07-06T01:19:37.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.277+0000 7f4bbd7fa700 1 -- 172.21.15.22:0/1274001721 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4bb4003a20 con 0x7f4bc00af2a0 2023-07-06T01:19:37.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.278+0000 7f4bbd7fa700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4ba406ace0 0x7f4ba406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:37.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.278+0000 7f4bbd7fa700 1 -- 172.21.15.22:0/1274001721 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4bb4088820 con 0x7f4bc00af2a0 2023-07-06T01:19:37.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.278+0000 7f4bbffff700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4ba406ace0 0x7f4ba406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:37.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.279+0000 7f4bbffff700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4ba406ace0 0x7f4ba406d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00ca740 tx=0x7f4bac008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:37.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.282+0000 7f4bbd7fa700 1 -- 172.21.15.22:0/1274001721 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4bb40538d0 con 0x7f4bc00af2a0 2023-07-06T01:19:37.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.525+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/1274001721 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bc0160020 con 0x7f4bc00af2a0 2023-07-06T01:19:37.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.526+0000 7f4bbd7fa700 1 -- 172.21.15.22:0/1274001721 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4671 (secure 0 0 0) 0x7f4bb4056ef0 con 0x7f4bc00af2a0 2023-07-06T01:19:37.529 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:37.529 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/2496681609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2496681609},{"type":"v1","addr":"172.21.15.22:6833","nonce":2496681609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2891594418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2891594418},{"type":"v1","addr":"172.21.15.22:6835","nonce":2891594418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/4097705589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4097705589},{"type":"v1","addr":"172.21.15.103:6836","nonce":4097705589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T01:19:25.042193+0000","modified":"2023-07-06T01:19:28.649724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6837/4015300665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":4015300665},{"type":"v1","addr":"172.21.15.103:6837","nonce":4015300665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T01:19:37.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.528+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4ba406ace0 msgr2=0x7f4ba406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.528+0000 7f4bc6b7a700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4ba406ace0 0x7f4ba406d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00ca740 tx=0x7f4bac008030 comp rx=0 tx=0).stop 2023-07-06T01:19:37.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.528+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/1274001721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00af2a0 msgr2=0x7f4bc015e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.528+0000 7f4bc6b7a700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00af2a0 0x7f4bc015e250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400c4e0 tx=0x7f4bb400c7f0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.528+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/1274001721 shutdown_connections 2023-07-06T01:19:37.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.528+0000 7f4bc6b7a700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4ba406ace0 0x7f4ba406d190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.528+0000 7f4bc6b7a700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bc01561d0 0x7f4bc00b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.528+0000 7f4bc6b7a700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4bc00af2a0 0x7f4bc015e250 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.528+0000 7f4bc6b7a700 1 --2- 172.21.15.22:0/1274001721 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bc00cbc10 0x7f4bc014cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.528+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/1274001721 >> 172.21.15.22:0/1274001721 conn(0x7f4bc00b1890 msgr2=0x7f4bc00cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:37.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.528+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/1274001721 shutdown_connections 2023-07-06T01:19:37.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.528+0000 7f4bc6b7a700 1 -- 172.21.15.22:0/1274001721 wait complete. 2023-07-06T01:19:37.533 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2023-07-06T01:19:37.543 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:37.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.665+0000 7fc83ecd0700 1 Processor -- start 2023-07-06T01:19:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.665+0000 7fc83ecd0700 1 -- start start 2023-07-06T01:19:37.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.665+0000 7fc83ecd0700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc83806b320 0x7fc83806c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.665+0000 7fc83ecd0700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc83806cb80 con 0x7fc83815a610 2023-07-06T01:19:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.665+0000 7fc83ecd0700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc838071f30 con 0x7fc83806c730 2023-07-06T01:19:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.665+0000 7fc83ecd0700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83815a9e0 con 0x7fc83806b320 2023-07-06T01:19:37.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc83ca6c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc83815a610 0x7fc83815eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36636/0 (socket says 172.21.15.22:36636) 2023-07-06T01:19:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc83ca6c700 1 -- 172.21.15.22:0/1186704205 learned_addr learned my addr 172.21.15.22:0/1186704205 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc837fff700 1 --2- 172.21.15.22:0/1186704205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc83806b320 0x7fc83806c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:37.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc8377fe700 1 -- 172.21.15.22:0/1186704205 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378681819 0 0) 0x7fc82c001110 con 0x7fc83815a610 2023-07-06T01:19:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc8377fe700 1 -- 172.21.15.22:0/1186704205 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8280035b0 con 0x7fc83815a610 2023-07-06T01:19:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc8377fe700 1 -- 172.21.15.22:0/1186704205 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143194640 0 0) 0x7fc838071f30 con 0x7fc83806c730 2023-07-06T01:19:37.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc8377fe700 1 -- 172.21.15.22:0/1186704205 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc82c001110 con 0x7fc83806c730 2023-07-06T01:19:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc837fff700 1 -- 172.21.15.22:0/1186704205 >> v1:172.21.15.103:6789/0 conn(0x7fc83806c730 legacy=0x7fc8380c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc837fff700 1 -- 172.21.15.22:0/1186704205 >> v1:172.21.15.22:6789/0 conn(0x7fc83815a610 legacy=0x7fc83815eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc837fff700 1 -- 172.21.15.22:0/1186704205 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83806b770 con 0x7fc83806b320 2023-07-06T01:19:37.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc837fff700 1 --2- 172.21.15.22:0/1186704205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc83806b320 0x7fc83806c1f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc824009870 tx=0x7fc824009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b7fc1b973109c0a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:37.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc8377fe700 1 -- 172.21.15.22:0/1186704205 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824011830 con 0x7fc83806b320 2023-07-06T01:19:37.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc8377fe700 1 -- 172.21.15.22:0/1186704205 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc824011990 con 0x7fc83806b320 2023-07-06T01:19:37.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.666+0000 7fc8377fe700 1 -- 172.21.15.22:0/1186704205 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824011d00 con 0x7fc83806b320 2023-07-06T01:19:37.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.667+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1186704205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc83806b320 msgr2=0x7fc83806c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.667+0000 7fc83ecd0700 1 --2- 172.21.15.22:0/1186704205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc83806b320 0x7fc83806c1f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc824009870 tx=0x7fc824009b80 comp rx=0 tx=0).stop 2023-07-06T01:19:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.667+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1186704205 shutdown_connections 2023-07-06T01:19:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.667+0000 7fc83ecd0700 1 --2- 172.21.15.22:0/1186704205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc83806b320 0x7fc83806c1f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.667+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1186704205 >> 172.21.15.22:0/1186704205 conn(0x7fc8380ccb00 msgr2=0x7fc8380cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:37.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.667+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1186704205 shutdown_connections 2023-07-06T01:19:37.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.667+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1186704205 wait complete. 2023-07-06T01:19:37.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.668+0000 7fc83ecd0700 1 Processor -- start 2023-07-06T01:19:37.679 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.668+0000 7fc83ecd0700 1 -- start start 2023-07-06T01:19:37.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ecd0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc83806b320 0x7fc838150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:37.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ecd0700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc83806c730 0x7fc8380ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:37.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ecd0700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc83815a610 0x7fc8380c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:37.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ecd0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc838151a80 con 0x7fc83806b320 2023-07-06T01:19:37.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ecd0700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380cea70 con 0x7fc83806c730 2023-07-06T01:19:37.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ecd0700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380c59e0 con 0x7fc83815a610 2023-07-06T01:19:37.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ca6c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc83806b320 0x7fc838150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:37.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ca6c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc83806b320 0x7fc838150700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45898/0 (socket says 172.21.15.22:45898) 2023-07-06T01:19:37.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ca6c700 1 -- 172.21.15.22:0/1316778006 learned_addr learned my addr 172.21.15.22:0/1316778006 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:37.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc837fff700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc83806c730 0x7fc8380ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:37.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ca6c700 1 -- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc83815a610 msgr2=0x7fc8380c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83d26d700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc83815a610 0x7fc8380c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:37.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ca6c700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc83815a610 0x7fc8380c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ca6c700 1 -- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc83806c730 msgr2=0x7fc8380ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ca6c700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc83806c730 0x7fc8380ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.669+0000 7fc83ca6c700 1 -- 172.21.15.22:0/1316778006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240094a0 con 0x7fc83806b320 2023-07-06T01:19:37.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.670+0000 7fc837fff700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc83806c730 0x7fc8380ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:37.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.670+0000 7fc83ca6c700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc83806b320 0x7fc838150700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00c680 tx=0x7fc82c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:37.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.670+0000 7fc8357fa700 1 -- 172.21.15.22:0/1316778006 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c005d70 con 0x7fc83806b320 2023-07-06T01:19:37.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.670+0000 7fc8357fa700 1 -- 172.21.15.22:0/1316778006 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82c007070 con 0x7fc83806b320 2023-07-06T01:19:37.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.670+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1316778006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc838150e00 con 0x7fc83806b320 2023-07-06T01:19:37.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.670+0000 7fc8357fa700 1 -- 172.21.15.22:0/1316778006 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c007420 con 0x7fc83806b320 2023-07-06T01:19:37.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.670+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1316778006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8381513c0 con 0x7fc83806b320 2023-07-06T01:19:37.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.671+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1316778006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc838152570 con 0x7fc83806b320 2023-07-06T01:19:37.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.671+0000 7fc8357fa700 1 -- 172.21.15.22:0/1316778006 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc82c007580 con 0x7fc83806b320 2023-07-06T01:19:37.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.672+0000 7fc8357fa700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc81c06ace0 0x7fc81c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:37.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.672+0000 7fc8357fa700 1 -- 172.21.15.22:0/1316778006 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc82c088810 con 0x7fc83806b320 2023-07-06T01:19:37.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.672+0000 7fc837fff700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc81c06ace0 0x7fc81c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:37.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.672+0000 7fc837fff700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc81c06ace0 0x7fc81c06d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc8380cee00 tx=0x7fc824008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:37.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.676+0000 7fc8357fa700 1 -- 172.21.15.22:0/1316778006 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc82c053950 con 0x7fc83806b320 2023-07-06T01:19:37.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.874+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1316778006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc83806cb80 con 0x7fc83806b320 2023-07-06T01:19:37.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.876+0000 7fc8357fa700 1 -- 172.21.15.22:0/1316778006 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7fc83806cb80 con 0x7fc83806b320 2023-07-06T01:19:37.879 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:37.880 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:37.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.878+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc81c06ace0 msgr2=0x7fc81c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.878+0000 7fc83ecd0700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc81c06ace0 0x7fc81c06d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc8380cee00 tx=0x7fc824008030 comp rx=0 tx=0).stop 2023-07-06T01:19:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.878+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1316778006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc83806b320 msgr2=0x7fc838150700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:37.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.878+0000 7fc83ecd0700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc83806b320 0x7fc838150700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00c680 tx=0x7fc82c00ca40 comp rx=0 tx=0).stop 2023-07-06T01:19:37.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.878+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1316778006 shutdown_connections 2023-07-06T01:19:37.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.878+0000 7fc83ecd0700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc83815a610 0x7fc8380c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.878+0000 7fc83ecd0700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc83806b320 0x7fc838150700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.878+0000 7fc83ecd0700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc81c06ace0 0x7fc81c06d190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.878+0000 7fc83ecd0700 1 --2- 172.21.15.22:0/1316778006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc83806c730 0x7fc8380ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:37.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.878+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1316778006 >> 172.21.15.22:0/1316778006 conn(0x7fc8380ccb00 msgr2=0x7fc83815bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:37.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.878+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1316778006 shutdown_connections 2023-07-06T01:19:37.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:37.878+0000 7fc83ecd0700 1 -- 172.21.15.22:0/1316778006 wait complete. 2023-07-06T01:19:37.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T01:19:22.873255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T01:19:24.019088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:19:37.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:19:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.018+0000 7efc8d019700 1 Processor -- start 2023-07-06T01:19:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.018+0000 7efc8d019700 1 -- start start 2023-07-06T01:19:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8d019700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7efc880af2a0 0x7efc8806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8d019700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc8806a000 0x7efc8806a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8d019700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efc880cbc10 0x7efc880cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8d019700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8806a9e0 con 0x7efc880cbc10 2023-07-06T01:19:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8d019700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc880af670 con 0x7efc880af2a0 2023-07-06T01:19:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8d019700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8806d030 con 0x7efc8806a000 2023-07-06T01:19:38.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8759e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efc880cbc10 0x7efc880cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8759e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efc880cbc10 0x7efc880cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45906/0 (socket says 172.21.15.22:45906) 2023-07-06T01:19:38.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8759e700 1 -- 172.21.15.22:0/2381415457 learned_addr learned my addr 172.21.15.22:0/2381415457 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:38.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc86d9d700 1 --2- 172.21.15.22:0/2381415457 >> v2:172.21.15.103:3300/0 conn(0x7efc880af2a0 0x7efc8806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8659c700 1 --2- 172.21.15.22:0/2381415457 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc8806a000 0x7efc8806a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8759e700 1 -- 172.21.15.22:0/2381415457 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc8806a000 msgr2=0x7efc8806a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:38.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8759e700 1 --2- 172.21.15.22:0/2381415457 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc8806a000 0x7efc8806a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8759e700 1 -- 172.21.15.22:0/2381415457 >> v2:172.21.15.103:3300/0 conn(0x7efc880af2a0 msgr2=0x7efc8806caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:38.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8759e700 1 --2- 172.21.15.22:0/2381415457 >> v2:172.21.15.103:3300/0 conn(0x7efc880af2a0 0x7efc8806caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8759e700 1 -- 172.21.15.22:0/2381415457 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc880cd560 con 0x7efc880cbc10 2023-07-06T01:19:38.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc86d9d700 1 --2- 172.21.15.22:0/2381415457 >> v2:172.21.15.103:3300/0 conn(0x7efc880af2a0 0x7efc8806caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:38.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.019+0000 7efc8659c700 1 --2- 172.21.15.22:0/2381415457 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc8806a000 0x7efc8806a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:38.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.020+0000 7efc8759e700 1 --2- 172.21.15.22:0/2381415457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc880cbc10 0x7efc880cc090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc88067460 tx=0x7efc8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989d5659a0ad6c1b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:38.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.020+0000 7efc85d9b700 1 -- 172.21.15.22:0/2381415457 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8000e070 con 0x7efc880cbc10 2023-07-06T01:19:38.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.020+0000 7efc85d9b700 1 -- 172.21.15.22:0/2381415457 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8000b980 con 0x7efc880cbc10 2023-07-06T01:19:38.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.020+0000 7efc85d9b700 1 -- 172.21.15.22:0/2381415457 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8000bd30 con 0x7efc880cbc10 2023-07-06T01:19:38.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.020+0000 7efc8d019700 1 -- 172.21.15.22:0/2381415457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc880cbc10 msgr2=0x7efc880cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.020+0000 7efc8d019700 1 --2- 172.21.15.22:0/2381415457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc880cbc10 0x7efc880cc090 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc88067460 tx=0x7efc8000b330 comp rx=0 tx=0).stop 2023-07-06T01:19:38.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.020+0000 7efc8d019700 1 -- 172.21.15.22:0/2381415457 shutdown_connections 2023-07-06T01:19:38.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.020+0000 7efc8d019700 1 --2- 172.21.15.22:0/2381415457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc880cbc10 0x7efc880cc090 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.020+0000 7efc8d019700 1 --2- 172.21.15.22:0/2381415457 >> v2:172.21.15.103:3300/0 conn(0x7efc880af2a0 0x7efc8806caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.020+0000 7efc8d019700 1 --2- 172.21.15.22:0/2381415457 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc8806a000 0x7efc8806a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.020+0000 7efc8d019700 1 -- 172.21.15.22:0/2381415457 >> 172.21.15.22:0/2381415457 conn(0x7efc880b1890 msgr2=0x7efc880b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:38.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.020+0000 7efc8d019700 1 -- 172.21.15.22:0/2381415457 shutdown_connections 2023-07-06T01:19:38.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.020+0000 7efc8d019700 1 -- 172.21.15.22:0/2381415457 wait complete. 2023-07-06T01:19:38.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.022+0000 7efc8d019700 1 Processor -- start 2023-07-06T01:19:38.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.022+0000 7efc8d019700 1 -- start start 2023-07-06T01:19:38.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8d019700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc8806a000 0x7efc8815a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8d019700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc880af2a0 0x7efc880c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8d019700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc880cbc10 0x7efc8806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8d019700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc880c9c60 con 0x7efc880af2a0 2023-07-06T01:19:38.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8d019700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc880c8680 con 0x7efc8806a000 2023-07-06T01:19:38.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8d019700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc880c8960 con 0x7efc880cbc10 2023-07-06T01:19:38.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc880af2a0 0x7efc880c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc880af2a0 0x7efc880c9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45912/0 (socket says 172.21.15.22:45912) 2023-07-06T01:19:38.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8659c700 1 -- 172.21.15.22:0/810632228 learned_addr learned my addr 172.21.15.22:0/810632228 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:38.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc86d9d700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc8806a000 0x7efc8815a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8759e700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc880cbc10 0x7efc8806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8659c700 1 -- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc880cbc10 msgr2=0x7efc8806d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8659c700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc880cbc10 0x7efc8806d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8659c700 1 -- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc8806a000 msgr2=0x7efc8815a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8659c700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc8806a000 0x7efc8815a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8659c700 1 -- 172.21.15.22:0/810632228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc80009b30 con 0x7efc880af2a0 2023-07-06T01:19:38.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8659c700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc880af2a0 0x7efc880c9720 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc7800e810 tx=0x7efc7800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:38.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc6f7fe700 1 -- 172.21.15.22:0/810632228 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7800c070 con 0x7efc880af2a0 2023-07-06T01:19:38.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc6f7fe700 1 -- 172.21.15.22:0/810632228 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc78004230 con 0x7efc880af2a0 2023-07-06T01:19:38.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc8d019700 1 -- 172.21.15.22:0/810632228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc880c8c40 con 0x7efc880af2a0 2023-07-06T01:19:38.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.023+0000 7efc6f7fe700 1 -- 172.21.15.22:0/810632228 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780036a0 con 0x7efc880af2a0 2023-07-06T01:19:38.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.024+0000 7efc8d019700 1 -- 172.21.15.22:0/810632228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc880c9200 con 0x7efc880af2a0 2023-07-06T01:19:38.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.025+0000 7efc8d019700 1 -- 172.21.15.22:0/810632228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc880c9570 con 0x7efc880af2a0 2023-07-06T01:19:38.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.025+0000 7efc6f7fe700 1 -- 172.21.15.22:0/810632228 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc78003800 con 0x7efc880af2a0 2023-07-06T01:19:38.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.025+0000 7efc6f7fe700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efc7406ace0 0x7efc7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.026+0000 7efc6f7fe700 1 -- 172.21.15.22:0/810632228 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efc78089020 con 0x7efc880af2a0 2023-07-06T01:19:38.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.026+0000 7efc86d9d700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efc7406ace0 0x7efc7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.034 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.026+0000 7efc86d9d700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efc7406ace0 0x7efc7406d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efc7c006f10 tx=0x7efc7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:38.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.030+0000 7efc6f7fe700 1 -- 172.21.15.22:0/810632228 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efc78054160 con 0x7efc880af2a0 2023-07-06T01:19:38.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.228+0000 7efc8d019700 1 -- 172.21.15.22:0/810632228 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efc8815ca90 con 0x7efc880af2a0 2023-07-06T01:19:38.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.228+0000 7efc6f7fe700 1 -- 172.21.15.22:0/810632228 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7efc8815ca90 con 0x7efc880af2a0 2023-07-06T01:19:38.232 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:19:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.230+0000 7efc8d019700 1 -- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efc7406ace0 msgr2=0x7efc7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.230+0000 7efc8d019700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efc7406ace0 0x7efc7406d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efc7c006f10 tx=0x7efc7c008040 comp rx=0 tx=0).stop 2023-07-06T01:19:38.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.230+0000 7efc8d019700 1 -- 172.21.15.22:0/810632228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc880af2a0 msgr2=0x7efc880c9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.230+0000 7efc8d019700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc880af2a0 0x7efc880c9720 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc7800e810 tx=0x7efc7800eb20 comp rx=0 tx=0).stop 2023-07-06T01:19:38.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.230+0000 7efc8d019700 1 -- 172.21.15.22:0/810632228 shutdown_connections 2023-07-06T01:19:38.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.230+0000 7efc8d019700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc880cbc10 0x7efc8806d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.230+0000 7efc8d019700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efc7406ace0 0x7efc7406d190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.230+0000 7efc8d019700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc8806a000 0x7efc8815a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.230+0000 7efc8d019700 1 --2- 172.21.15.22:0/810632228 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc880af2a0 0x7efc880c9720 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.230+0000 7efc8d019700 1 -- 172.21.15.22:0/810632228 >> 172.21.15.22:0/810632228 conn(0x7efc880b1890 msgr2=0x7efc880b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:38.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.230+0000 7efc8d019700 1 -- 172.21.15.22:0/810632228 shutdown_connections 2023-07-06T01:19:38.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.230+0000 7efc8d019700 1 -- 172.21.15.22:0/810632228 wait complete. 2023-07-06T01:19:38.245 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:19:38.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.370+0000 7f2991fea700 1 Processor -- start 2023-07-06T01:19:38.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.370+0000 7f2991fea700 1 -- start start 2023-07-06T01:19:38.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.371+0000 7f2991fea700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f298c159360 0x7f298c15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.371+0000 7f2991fea700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f298c0cc660 0x7f298c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.371+0000 7f2991fea700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298c0cf230 con 0x7f298c070a20 2023-07-06T01:19:38.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.371+0000 7f2991fea700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298c15df20 con 0x7f298c159360 2023-07-06T01:19:38.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.371+0000 7f2991fea700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298c0ccff0 con 0x7f298c0cc660 2023-07-06T01:19:38.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.371+0000 7f298b7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f298c159360 0x7f298c15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.371+0000 7f298b7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f298c159360 0x7f298c15d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45918/0 (socket says 172.21.15.22:45918) 2023-07-06T01:19:38.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.371+0000 7f298b7fe700 1 -- 172.21.15.22:0/3682676015 learned_addr learned my addr 172.21.15.22:0/3682676015 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:38.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.371+0000 7f298affd700 1 --2- 172.21.15.22:0/3682676015 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f298c0cc660 0x7f298c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.372+0000 7f298b7fe700 1 -- 172.21.15.22:0/3682676015 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f298c0cc660 msgr2=0x7f298c0ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:38.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.372+0000 7f298b7fe700 1 --2- 172.21.15.22:0/3682676015 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f298c0cc660 0x7f298c0ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.372+0000 7f298b7fe700 1 -- 172.21.15.22:0/3682676015 >> v1:172.21.15.103:6789/0 conn(0x7f298c070a20 legacy=0x7f298c070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.372+0000 7f298b7fe700 1 -- 172.21.15.22:0/3682676015 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c0ce620 con 0x7f298c159360 2023-07-06T01:19:38.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.372+0000 7f298b7fe700 1 --2- 172.21.15.22:0/3682676015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f298c159360 0x7f298c15d9e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f297c0098f0 tx=0x7f297c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89680e46f764ce03 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:38.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.372+0000 7f298a7fc700 1 -- 172.21.15.22:0/3682676015 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c00b8a0 con 0x7f298c159360 2023-07-06T01:19:38.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.372+0000 7f298a7fc700 1 -- 172.21.15.22:0/3682676015 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c00ba00 con 0x7f298c159360 2023-07-06T01:19:38.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.372+0000 7f298a7fc700 1 -- 172.21.15.22:0/3682676015 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c00bd90 con 0x7f298c159360 2023-07-06T01:19:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.372+0000 7f2991fea700 1 -- 172.21.15.22:0/3682676015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f298c159360 msgr2=0x7f298c15d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.372+0000 7f2991fea700 1 --2- 172.21.15.22:0/3682676015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f298c159360 0x7f298c15d9e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f297c0098f0 tx=0x7f297c009c00 comp rx=0 tx=0).stop 2023-07-06T01:19:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.373+0000 7f2991fea700 1 -- 172.21.15.22:0/3682676015 shutdown_connections 2023-07-06T01:19:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.373+0000 7f2991fea700 1 --2- 172.21.15.22:0/3682676015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f298c159360 0x7f298c15d9e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.373+0000 7f2991fea700 1 --2- 172.21.15.22:0/3682676015 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f298c0cc660 0x7f298c0ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.373+0000 7f2991fea700 1 -- 172.21.15.22:0/3682676015 >> 172.21.15.22:0/3682676015 conn(0x7f298c0b1890 msgr2=0x7f298c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.373+0000 7f2991fea700 1 -- 172.21.15.22:0/3682676015 shutdown_connections 2023-07-06T01:19:38.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.373+0000 7f2991fea700 1 -- 172.21.15.22:0/3682676015 wait complete. 2023-07-06T01:19:38.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.374+0000 7f2991fea700 1 Processor -- start 2023-07-06T01:19:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.374+0000 7f2991fea700 1 -- start start 2023-07-06T01:19:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f2991fea700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f298c070a20 0x7f298c06e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f2991fea700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f298c0cc660 0x7f298c1f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f2991fea700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f298c159360 0x7f298c1f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f2991fea700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c1f35c0 con 0x7f298c070a20 2023-07-06T01:19:38.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f2991fea700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c1f3f10 con 0x7f298c159360 2023-07-06T01:19:38.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f2991fea700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298c1f32e0 con 0x7f298c0cc660 2023-07-06T01:19:38.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f298b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f298c070a20 0x7f298c06e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f298b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f298c070a20 0x7f298c06e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45924/0 (socket says 172.21.15.22:45924) 2023-07-06T01:19:38.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f298b7fe700 1 -- 172.21.15.22:0/639827296 learned_addr learned my addr 172.21.15.22:0/639827296 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:38.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f298bfff700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f298c159360 0x7f298c1f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f298affd700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f298c0cc660 0x7f298c1f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f298b7fe700 1 -- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f298c0cc660 msgr2=0x7f298c1f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f298b7fe700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f298c0cc660 0x7f298c1f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f298b7fe700 1 -- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f298c159360 msgr2=0x7f298c1f2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f298b7fe700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f298c159360 0x7f298c1f2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f298b7fe700 1 -- 172.21.15.22:0/639827296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c009550 con 0x7f298c070a20 2023-07-06T01:19:38.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f298bfff700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f298c159360 0x7f298c1f2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:38.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f298affd700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f298c0cc660 0x7f298c1f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:38.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.375+0000 7f298b7fe700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f298c070a20 0x7f298c06e2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f297c009c40 tx=0x7f297c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:38.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.376+0000 7f2973fff700 1 -- 172.21.15.22:0/639827296 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c0036a0 con 0x7f298c070a20 2023-07-06T01:19:38.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.376+0000 7f2973fff700 1 -- 172.21.15.22:0/639827296 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c01e070 con 0x7f298c070a20 2023-07-06T01:19:38.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.376+0000 7f2991fea700 1 -- 172.21.15.22:0/639827296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298c1f38c0 con 0x7f298c070a20 2023-07-06T01:19:38.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.376+0000 7f2973fff700 1 -- 172.21.15.22:0/639827296 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c003a60 con 0x7f298c070a20 2023-07-06T01:19:38.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.376+0000 7f2991fea700 1 -- 172.21.15.22:0/639827296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298c1f2fe0 con 0x7f298c070a20 2023-07-06T01:19:38.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.377+0000 7f2991fea700 1 -- 172.21.15.22:0/639827296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298c1f3720 con 0x7f298c070a20 2023-07-06T01:19:38.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.377+0000 7f2973fff700 1 -- 172.21.15.22:0/639827296 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f297c003bc0 con 0x7f298c070a20 2023-07-06T01:19:38.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.377+0000 7f2973fff700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f297806ace0 0x7f297806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.378+0000 7f2973fff700 1 -- 172.21.15.22:0/639827296 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f297c020070 con 0x7f298c070a20 2023-07-06T01:19:38.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.378+0000 7f298affd700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f297806ace0 0x7f297806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.378+0000 7f298affd700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f297806ace0 0x7f297806d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f298c15c8a0 tx=0x7f2980008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:38.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.382+0000 7f2973fff700 1 -- 172.21.15.22:0/639827296 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f298c1f3720 con 0x7f298c070a20 2023-07-06T01:19:38.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.577+0000 7f2991fea700 1 -- 172.21.15.22:0/639827296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f298c1f3720 con 0x7f298c070a20 2023-07-06T01:19:38.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.578+0000 7f2973fff700 1 -- 172.21.15.22:0/639827296 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f298c1f3720 con 0x7f298c070a20 2023-07-06T01:19:38.581 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:19:38.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.580+0000 7f2991fea700 1 -- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f297806ace0 msgr2=0x7f297806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.580+0000 7f2991fea700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f297806ace0 0x7f297806d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f298c15c8a0 tx=0x7f2980008040 comp rx=0 tx=0).stop 2023-07-06T01:19:38.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.580+0000 7f2991fea700 1 -- 172.21.15.22:0/639827296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f298c070a20 msgr2=0x7f298c06e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.580+0000 7f2991fea700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f298c070a20 0x7f298c06e2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f297c009c40 tx=0x7f297c004750 comp rx=0 tx=0).stop 2023-07-06T01:19:38.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.580+0000 7f2991fea700 1 -- 172.21.15.22:0/639827296 shutdown_connections 2023-07-06T01:19:38.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.580+0000 7f2991fea700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f297806ace0 0x7f297806d190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.580+0000 7f2991fea700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f298c159360 0x7f298c1f2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.580+0000 7f2991fea700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f298c070a20 0x7f298c06e2c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.580+0000 7f2991fea700 1 --2- 172.21.15.22:0/639827296 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f298c0cc660 0x7f298c1f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.580+0000 7f2991fea700 1 -- 172.21.15.22:0/639827296 >> 172.21.15.22:0/639827296 conn(0x7f298c0b1890 msgr2=0x7f298c0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.580+0000 7f2991fea700 1 -- 172.21.15.22:0/639827296 shutdown_connections 2023-07-06T01:19:38.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.580+0000 7f2991fea700 1 -- 172.21.15.22:0/639827296 wait complete. 2023-07-06T01:19:38.594 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T01:19:38.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.719+0000 7f1618571700 1 Processor -- start 2023-07-06T01:19:38.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.719+0000 7f1618571700 1 -- start start 2023-07-06T01:19:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.720+0000 7f1618571700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16100ccd70 0x7f161015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.720+0000 7f1618571700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16100cf230 con 0x7f1610159360 2023-07-06T01:19:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.720+0000 7f1618571700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16100ce620 con 0x7f161006d040 2023-07-06T01:19:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.720+0000 7f1618571700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161015df20 con 0x7f16100ccd70 2023-07-06T01:19:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.720+0000 7f161630d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1610159360 0x7f16100cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36676/0 (socket says 172.21.15.22:36676) 2023-07-06T01:19:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.720+0000 7f161630d700 1 -- 172.21.15.22:0/2704728594 learned_addr learned my addr 172.21.15.22:0/2704728594 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f1615b0c700 1 --2- 172.21.15.22:0/2704728594 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16100ccd70 0x7f161015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496882140 0 0) 0x7f16100cf230 con 0x7f1610159360 2023-07-06T01:19:38.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15fc0035b0 con 0x7f1610159360 2023-07-06T01:19:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 970833594 0 0) 0x7f15fc0035b0 con 0x7f1610159360 2023-07-06T01:19:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16100ccd70 msgr2=0x7f161015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 --2- 172.21.15.22:0/2704728594 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16100ccd70 0x7f161015d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 >> v1:172.21.15.103:6789/0 conn(0x7f161006d040 legacy=0x7f161006d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16100ce900 con 0x7f1610159360 2023-07-06T01:19:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f1600001670 con 0x7f1610159360 2023-07-06T01:19:38.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 >> v1:172.21.15.22:6789/0 conn(0x7f1610159360 legacy=0x7f16100cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 --2- 172.21.15.22:0/2704728594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15fc003a30 0x7f15fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 --2- 172.21.15.22:0/2704728594 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16100ccd70 0x7f15fc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 --2- 172.21.15.22:0/2704728594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15fc004d90 0x7f15fc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161015df20 con 0x7f15fc003a30 2023-07-06T01:19:38.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc00ba10 con 0x7f15fc004d90 2023-07-06T01:19:38.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc00ddb0 con 0x7f16100ccd70 2023-07-06T01:19:38.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f161630d700 1 --2- 172.21.15.22:0/2704728594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15fc003a30 0x7f15fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f1615b0c700 1 --2- 172.21.15.22:0/2704728594 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16100ccd70 0x7f15fc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.721+0000 7f1616b0e700 1 --2- 172.21.15.22:0/2704728594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15fc004d90 0x7f15fc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f161630d700 1 -- 172.21.15.22:0/2704728594 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16100ccd70 msgr2=0x7f15fc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:38.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f161630d700 1 --2- 172.21.15.22:0/2704728594 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16100ccd70 0x7f15fc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f161630d700 1 -- 172.21.15.22:0/2704728594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15fc004d90 msgr2=0x7f15fc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:38.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f161630d700 1 --2- 172.21.15.22:0/2704728594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15fc004d90 0x7f15fc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f161630d700 1 -- 172.21.15.22:0/2704728594 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc0035b0 con 0x7f15fc003a30 2023-07-06T01:19:38.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f161630d700 1 --2- 172.21.15.22:0/2704728594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15fc003a30 0x7f15fc003e00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f160000c2f0 tx=0x7f15fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d23c2fa391ceb74 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:38.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160000ce30 con 0x7f15fc003a30 2023-07-06T01:19:38.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1600005960 con 0x7f15fc003a30 2023-07-06T01:19:38.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f161530b700 1 -- 172.21.15.22:0/2704728594 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600005cf0 con 0x7f15fc003a30 2023-07-06T01:19:38.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f1618571700 1 -- 172.21.15.22:0/2704728594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15fc003a30 msgr2=0x7f15fc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f1618571700 1 --2- 172.21.15.22:0/2704728594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15fc003a30 0x7f15fc003e00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f160000c2f0 tx=0x7f15fc005880 comp rx=0 tx=0).stop 2023-07-06T01:19:38.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f1616b0e700 1 -- 172.21.15.22:0/2704728594 reap_dead start 2023-07-06T01:19:38.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f1618571700 1 -- 172.21.15.22:0/2704728594 shutdown_connections 2023-07-06T01:19:38.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f1618571700 1 -- 172.21.15.22:0/2704728594 >> 172.21.15.22:0/2704728594 conn(0x7f16100b1890 msgr2=0x7f16100b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:38.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f1618571700 1 -- 172.21.15.22:0/2704728594 shutdown_connections 2023-07-06T01:19:38.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.722+0000 7f1618571700 1 -- 172.21.15.22:0/2704728594 wait complete. 2023-07-06T01:19:38.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.724+0000 7f1618571700 1 Processor -- start 2023-07-06T01:19:38.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.724+0000 7f1618571700 1 -- start start 2023-07-06T01:19:38.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.724+0000 7f1618571700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15fc003a30 0x7f161015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.724+0000 7f1618571700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f161014be80 0x7f16100ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.724+0000 7f1618571700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16100cae80 0x7f16100b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.724+0000 7f1618571700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16100b48e0 con 0x7f16100cae80 2023-07-06T01:19:38.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.724+0000 7f1618571700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16100b3c60 con 0x7f161014be80 2023-07-06T01:19:38.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.724+0000 7f1618571700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16100b3f40 con 0x7f15fc003a30 2023-07-06T01:19:38.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.724+0000 7f1616b0e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16100cae80 0x7f16100b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1616b0e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16100cae80 0x7f16100b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45948/0 (socket says 172.21.15.22:45948) 2023-07-06T01:19:38.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1616b0e700 1 -- 172.21.15.22:0/923959047 learned_addr learned my addr 172.21.15.22:0/923959047 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:38.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1615b0c700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f161014be80 0x7f16100ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f161630d700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15fc003a30 0x7f161015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1616b0e700 1 -- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15fc003a30 msgr2=0x7f161015c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1616b0e700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15fc003a30 0x7f161015c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1616b0e700 1 -- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f161014be80 msgr2=0x7f16100ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1616b0e700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f161014be80 0x7f16100ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1616b0e700 1 -- 172.21.15.22:0/923959047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16100b4220 con 0x7f16100cae80 2023-07-06T01:19:38.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1616b0e700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16100cae80 0x7f16100b4d00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f16100cdd90 tx=0x7f160800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:38.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1606ffd700 1 -- 172.21.15.22:0/923959047 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16080020e0 con 0x7f16100cae80 2023-07-06T01:19:38.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1606ffd700 1 -- 172.21.15.22:0/923959047 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1608003c10 con 0x7f16100cae80 2023-07-06T01:19:38.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1606ffd700 1 -- 172.21.15.22:0/923959047 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160800d990 con 0x7f16100cae80 2023-07-06T01:19:38.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1618571700 1 -- 172.21.15.22:0/923959047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16100b4500 con 0x7f16100cae80 2023-07-06T01:19:38.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.725+0000 7f1618571700 1 -- 172.21.15.22:0/923959047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16100b5240 con 0x7f16100cae80 2023-07-06T01:19:38.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.726+0000 7f1618571700 1 -- 172.21.15.22:0/923959047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16100b4a20 con 0x7f16100cae80 2023-07-06T01:19:38.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.727+0000 7f1606ffd700 1 -- 172.21.15.22:0/923959047 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f160800daf0 con 0x7f16100cae80 2023-07-06T01:19:38.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.727+0000 7f1606ffd700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f15f406ace0 0x7f15f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:38.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.727+0000 7f1606ffd700 1 -- 172.21.15.22:0/923959047 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1608089680 con 0x7f16100cae80 2023-07-06T01:19:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.727+0000 7f161630d700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f15f406ace0 0x7f15f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.728+0000 7f161630d700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f15f406ace0 0x7f15f406d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1600001930 tx=0x7f1600007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.732+0000 7f1606ffd700 1 -- 172.21.15.22:0/923959047 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f16100b4a20 con 0x7f16100cae80 2023-07-06T01:19:38.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.927+0000 7f1618571700 1 -- 172.21.15.22:0/923959047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f16100b4a20 con 0x7f16100cae80 2023-07-06T01:19:38.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.928+0000 7f1606ffd700 1 -- 172.21.15.22:0/923959047 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f16100b4a20 con 0x7f16100cae80 2023-07-06T01:19:38.931 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-07-06T01:19:38.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.930+0000 7f1618571700 1 -- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f15f406ace0 msgr2=0x7f15f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.930+0000 7f1618571700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f15f406ace0 0x7f15f406d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1600001930 tx=0x7f1600007000 comp rx=0 tx=0).stop 2023-07-06T01:19:38.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.930+0000 7f1618571700 1 -- 172.21.15.22:0/923959047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16100cae80 msgr2=0x7f16100b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:38.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.930+0000 7f1618571700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16100cae80 0x7f16100b4d00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f16100cdd90 tx=0x7f160800d430 comp rx=0 tx=0).stop 2023-07-06T01:19:38.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.930+0000 7f1618571700 1 -- 172.21.15.22:0/923959047 shutdown_connections 2023-07-06T01:19:38.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.930+0000 7f1618571700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16100cae80 0x7f16100b4d00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.930+0000 7f1618571700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15fc003a30 0x7f161015c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.930+0000 7f1618571700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f15f406ace0 0x7f15f406d190 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.930+0000 7f1618571700 1 --2- 172.21.15.22:0/923959047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f161014be80 0x7f16100ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:38.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.930+0000 7f1618571700 1 -- 172.21.15.22:0/923959047 >> 172.21.15.22:0/923959047 conn(0x7f16100b1890 msgr2=0x7f16100712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:38.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.930+0000 7f1618571700 1 -- 172.21.15.22:0/923959047 shutdown_connections 2023-07-06T01:19:38.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:38.930+0000 7f1618571700 1 -- 172.21.15.22:0/923959047 wait complete. 2023-07-06T01:19:38.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T01:19:39.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.069+0000 7f66ffe1a700 1 Processor -- start 2023-07-06T01:19:39.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.069+0000 7f66ffe1a700 1 -- start start 2023-07-06T01:19:39.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.070+0000 7f66ffe1a700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f66f80d0300 0x7f66f8164720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.070+0000 7f66ffe1a700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f80ce550 con 0x7f66f8159870 2023-07-06T01:19:39.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.070+0000 7f66ffe1a700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f80cdb60 con 0x7f66f815df70 2023-07-06T01:19:39.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.070+0000 7f66ffe1a700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f815d490 con 0x7f66f80d0300 2023-07-06T01:19:39.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.070+0000 7f66fdbb6700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f66f8159870 0x7f66f80715f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36696/0 (socket says 172.21.15.22:36696) 2023-07-06T01:19:39.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.070+0000 7f66fdbb6700 1 -- 172.21.15.22:0/2254031767 learned_addr learned my addr 172.21.15.22:0/2254031767 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:39.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.070+0000 7f66fe3b7700 1 --2- 172.21.15.22:0/2254031767 >> v2:172.21.15.103:3300/0 conn(0x7f66f80d0300 0x7f66f8164720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.070+0000 7f66fcbb4700 1 -- 172.21.15.22:0/2254031767 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749355514 0 0) 0x7f66f80ce550 con 0x7f66f8159870 2023-07-06T01:19:39.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.070+0000 7f66fcbb4700 1 -- 172.21.15.22:0/2254031767 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66e40035b0 con 0x7f66f8159870 2023-07-06T01:19:39.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.070+0000 7f66fe3b7700 1 -- 172.21.15.22:0/2254031767 >> v1:172.21.15.103:6789/0 conn(0x7f66f815df70 legacy=0x7f66f80cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.070+0000 7f66fe3b7700 1 -- 172.21.15.22:0/2254031767 >> v1:172.21.15.22:6789/0 conn(0x7f66f8159870 legacy=0x7f66f80715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.070+0000 7f66fe3b7700 1 -- 172.21.15.22:0/2254031767 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f815d770 con 0x7f66f80d0300 2023-07-06T01:19:39.076 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.070+0000 7f66fe3b7700 1 --2- 172.21.15.22:0/2254031767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66f80d0300 0x7f66f8164720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009d90 tx=0x7f66f000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8f488e798c1daeb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:39.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.071+0000 7f66fcbb4700 1 -- 172.21.15.22:0/2254031767 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f000e070 con 0x7f66f80d0300 2023-07-06T01:19:39.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.071+0000 7f66fcbb4700 1 -- 172.21.15.22:0/2254031767 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f000bb20 con 0x7f66f80d0300 2023-07-06T01:19:39.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.071+0000 7f66fcbb4700 1 -- 172.21.15.22:0/2254031767 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f000be90 con 0x7f66f80d0300 2023-07-06T01:19:39.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.071+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2254031767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66f80d0300 msgr2=0x7f66f8164720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.071+0000 7f66ffe1a700 1 --2- 172.21.15.22:0/2254031767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66f80d0300 0x7f66f8164720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009d90 tx=0x7f66f000b330 comp rx=0 tx=0).stop 2023-07-06T01:19:39.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.071+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2254031767 shutdown_connections 2023-07-06T01:19:39.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.071+0000 7f66ffe1a700 1 --2- 172.21.15.22:0/2254031767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66f80d0300 0x7f66f8164720 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.071+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2254031767 >> 172.21.15.22:0/2254031767 conn(0x7f66f806b110 msgr2=0x7f66f806b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:39.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.071+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2254031767 shutdown_connections 2023-07-06T01:19:39.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.071+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2254031767 wait complete. 2023-07-06T01:19:39.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.073+0000 7f66ffe1a700 1 Processor -- start 2023-07-06T01:19:39.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.073+0000 7f66ffe1a700 1 -- start start 2023-07-06T01:19:39.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66ffe1a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66f80d0300 0x7f66f80c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66ffe1a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66f8159870 0x7f66f81f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66ffe1a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f815df70 0x7f66f8151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66ffe1a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f8148d60 con 0x7f66f815df70 2023-07-06T01:19:39.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66ffe1a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f81f1090 con 0x7f66f8159870 2023-07-06T01:19:39.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66ffe1a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f81f0ad0 con 0x7f66f80d0300 2023-07-06T01:19:39.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66fe3b7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f815df70 0x7f66f8151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66fe3b7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f815df70 0x7f66f8151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45962/0 (socket says 172.21.15.22:45962) 2023-07-06T01:19:39.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66fe3b7700 1 -- 172.21.15.22:0/2690019211 learned_addr learned my addr 172.21.15.22:0/2690019211 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:39.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66fdbb6700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66f80d0300 0x7f66f80c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66fd3b5700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66f8159870 0x7f66f81f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66fe3b7700 1 -- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66f80d0300 msgr2=0x7f66f80c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66fe3b7700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66f80d0300 0x7f66f80c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66fe3b7700 1 -- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66f8159870 msgr2=0x7f66f81f00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66fe3b7700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66f8159870 0x7f66f81f00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66fe3b7700 1 -- 172.21.15.22:0/2690019211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f0009a70 con 0x7f66f815df70 2023-07-06T01:19:39.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66fdbb6700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66f80d0300 0x7f66f80c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:39.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66fd3b5700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66f8159870 0x7f66f81f00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:39.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.074+0000 7f66fe3b7700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f815df70 0x7f66f8151db0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f66f000b370 tx=0x7f66f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.075+0000 7f66ee7fc700 1 -- 172.21.15.22:0/2690019211 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0004960 con 0x7f66f815df70 2023-07-06T01:19:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.075+0000 7f66ee7fc700 1 -- 172.21.15.22:0/2690019211 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f0020070 con 0x7f66f815df70 2023-07-06T01:19:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.075+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2690019211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f81f0db0 con 0x7f66f815df70 2023-07-06T01:19:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.075+0000 7f66ee7fc700 1 -- 172.21.15.22:0/2690019211 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f00276c0 con 0x7f66f815df70 2023-07-06T01:19:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.075+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2690019211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f80b5710 con 0x7f66f815df70 2023-07-06T01:19:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.076+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2690019211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f80b5930 con 0x7f66f815df70 2023-07-06T01:19:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.076+0000 7f66ee7fc700 1 -- 172.21.15.22:0/2690019211 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f66f0012430 con 0x7f66f815df70 2023-07-06T01:19:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.076+0000 7f66ee7fc700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f66dc06ace0 0x7f66dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.077+0000 7f66ee7fc700 1 -- 172.21.15.22:0/2690019211 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f66f0022070 con 0x7f66f815df70 2023-07-06T01:19:39.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.077+0000 7f66fdbb6700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f66dc06ace0 0x7f66dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.077+0000 7f66fdbb6700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f66dc06ace0 0x7f66dc06d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f66e80011e0 tx=0x7f66e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:39.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.081+0000 7f66ee7fc700 1 -- 172.21.15.22:0/2690019211 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f66f80b5930 con 0x7f66f815df70 2023-07-06T01:19:39.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.278+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2690019211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f66f80b5930 con 0x7f66f815df70 2023-07-06T01:19:39.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.278+0000 7f66ee7fc700 1 -- 172.21.15.22:0/2690019211 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f66f80b5930 con 0x7f66f815df70 2023-07-06T01:19:39.281 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-07-06T01:19:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.280+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f66dc06ace0 msgr2=0x7f66dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.280+0000 7f66ffe1a700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f66dc06ace0 0x7f66dc06d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f66e80011e0 tx=0x7f66e8009040 comp rx=0 tx=0).stop 2023-07-06T01:19:39.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.280+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2690019211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f815df70 msgr2=0x7f66f8151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.280+0000 7f66ffe1a700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f815df70 0x7f66f8151db0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f66f000b370 tx=0x7f66f0004ad0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.280+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2690019211 shutdown_connections 2023-07-06T01:19:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.280+0000 7f66ffe1a700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66f815df70 0x7f66f8151db0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.280+0000 7f66ffe1a700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66f80d0300 0x7f66f80c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.280+0000 7f66ffe1a700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f66dc06ace0 0x7f66dc06d190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.280+0000 7f66ffe1a700 1 --2- 172.21.15.22:0/2690019211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66f8159870 0x7f66f81f00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.280+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2690019211 >> 172.21.15.22:0/2690019211 conn(0x7f66f806b110 msgr2=0x7f66f81538e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.280+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2690019211 shutdown_connections 2023-07-06T01:19:39.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.280+0000 7f66ffe1a700 1 -- 172.21.15.22:0/2690019211 wait complete. 2023-07-06T01:19:39.295 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:19:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.417+0000 7f1a99b99700 1 Processor -- start 2023-07-06T01:19:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.418+0000 7f1a99b99700 1 -- start start 2023-07-06T01:19:39.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.418+0000 7f1a99b99700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f1a940ccd70 0x7f1a9415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.418+0000 7f1a99b99700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1a9406d040 0x7f1a9406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.418+0000 7f1a99b99700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a940cf230 con 0x7f1a94159360 2023-07-06T01:19:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.418+0000 7f1a99b99700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9415df20 con 0x7f1a9406d040 2023-07-06T01:19:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.418+0000 7f1a99b99700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a940ce620 con 0x7f1a940ccd70 2023-07-06T01:19:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.418+0000 7f1a93fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1a9406d040 0x7f1a9406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.418+0000 7f1a937fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1a94159360 0x7f1a940cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36710/0 (socket says 172.21.15.22:36710) 2023-07-06T01:19:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.418+0000 7f1a937fe700 1 -- 172.21.15.22:0/2885213659 learned_addr learned my addr 172.21.15.22:0/2885213659 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.419+0000 7f1a92ffd700 1 --2- 172.21.15.22:0/2885213659 >> v2:172.21.15.103:3300/0 conn(0x7f1a940ccd70 0x7f1a9415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.419+0000 7f1a93fff700 1 -- 172.21.15.22:0/2885213659 >> v2:172.21.15.103:3300/0 conn(0x7f1a940ccd70 msgr2=0x7f1a9415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.419+0000 7f1a93fff700 1 --2- 172.21.15.22:0/2885213659 >> v2:172.21.15.103:3300/0 conn(0x7f1a940ccd70 0x7f1a9415d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.419+0000 7f1a93fff700 1 -- 172.21.15.22:0/2885213659 >> v1:172.21.15.22:6789/0 conn(0x7f1a94159360 legacy=0x7f1a940cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.419+0000 7f1a93fff700 1 -- 172.21.15.22:0/2885213659 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a940ce900 con 0x7f1a9406d040 2023-07-06T01:19:39.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.419+0000 7f1a927fc700 1 -- 172.21.15.22:0/2885213659 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380056164 0 0) 0x7f1a940cf230 con 0x7f1a94159360 2023-07-06T01:19:39.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.419+0000 7f1a93fff700 1 --2- 172.21.15.22:0/2885213659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a9406d040 0x7f1a9406d4a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1a940af7d0 tx=0x7f1a8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6e7af814dd78ac server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.419+0000 7f1a927fc700 1 -- 172.21.15.22:0/2885213659 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c00e070 con 0x7f1a9406d040 2023-07-06T01:19:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.419+0000 7f1a927fc700 1 -- 172.21.15.22:0/2885213659 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c00bbd0 con 0x7f1a9406d040 2023-07-06T01:19:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.419+0000 7f1a927fc700 1 -- 172.21.15.22:0/2885213659 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c012430 con 0x7f1a9406d040 2023-07-06T01:19:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.420+0000 7f1a99b99700 1 -- 172.21.15.22:0/2885213659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a9406d040 msgr2=0x7f1a9406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.420+0000 7f1a99b99700 1 --2- 172.21.15.22:0/2885213659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a9406d040 0x7f1a9406d4a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1a940af7d0 tx=0x7f1a8c00b330 comp rx=0 tx=0).stop 2023-07-06T01:19:39.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.420+0000 7f1a99b99700 1 -- 172.21.15.22:0/2885213659 shutdown_connections 2023-07-06T01:19:39.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.420+0000 7f1a99b99700 1 --2- 172.21.15.22:0/2885213659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a9406d040 0x7f1a9406d4a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.420+0000 7f1a99b99700 1 --2- 172.21.15.22:0/2885213659 >> v2:172.21.15.103:3300/0 conn(0x7f1a940ccd70 0x7f1a9415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.420+0000 7f1a99b99700 1 -- 172.21.15.22:0/2885213659 >> 172.21.15.22:0/2885213659 conn(0x7f1a940b1890 msgr2=0x7f1a940b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:39.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.420+0000 7f1a99b99700 1 -- 172.21.15.22:0/2885213659 shutdown_connections 2023-07-06T01:19:39.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.420+0000 7f1a99b99700 1 -- 172.21.15.22:0/2885213659 wait complete. 2023-07-06T01:19:39.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.421+0000 7f1a99b99700 1 Processor -- start 2023-07-06T01:19:39.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.421+0000 7f1a99b99700 1 -- start start 2023-07-06T01:19:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.422+0000 7f1a99b99700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a9406d040 0x7f1a940c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.422+0000 7f1a99b99700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a940ccd70 0x7f1a940c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.422+0000 7f1a99b99700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a94159360 0x7f1a940c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.422+0000 7f1a99b99700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940c9cf0 con 0x7f1a940ccd70 2023-07-06T01:19:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.422+0000 7f1a99b99700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940c7420 con 0x7f1a94159360 2023-07-06T01:19:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.422+0000 7f1a99b99700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940c6e10 con 0x7f1a9406d040 2023-07-06T01:19:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a92ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a940ccd70 0x7f1a940c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a92ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a940ccd70 0x7f1a940c8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45974/0 (socket says 172.21.15.22:45974) 2023-07-06T01:19:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a92ffd700 1 -- 172.21.15.22:0/2359112118 learned_addr learned my addr 172.21.15.22:0/2359112118 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a937fe700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a9406d040 0x7f1a940c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a93fff700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a94159360 0x7f1a940c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a92ffd700 1 -- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a9406d040 msgr2=0x7f1a940c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a92ffd700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a9406d040 0x7f1a940c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a92ffd700 1 -- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a94159360 msgr2=0x7f1a940c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a92ffd700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a94159360 0x7f1a940c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a92ffd700 1 -- 172.21.15.22:0/2359112118 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c009a70 con 0x7f1a940ccd70 2023-07-06T01:19:39.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a92ffd700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a940ccd70 0x7f1a940c8550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1a880106b0 tx=0x7f1a88010a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:39.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a7bfff700 1 -- 172.21.15.22:0/2359112118 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88011920 con 0x7f1a940ccd70 2023-07-06T01:19:39.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a7bfff700 1 -- 172.21.15.22:0/2359112118 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a88011a80 con 0x7f1a940ccd70 2023-07-06T01:19:39.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a7bfff700 1 -- 172.21.15.22:0/2359112118 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88011e10 con 0x7f1a940ccd70 2023-07-06T01:19:39.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a99b99700 1 -- 172.21.15.22:0/2359112118 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a940b4b60 con 0x7f1a940ccd70 2023-07-06T01:19:39.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.423+0000 7f1a99b99700 1 -- 172.21.15.22:0/2359112118 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a940b4e40 con 0x7f1a940ccd70 2023-07-06T01:19:39.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.424+0000 7f1a99b99700 1 -- 172.21.15.22:0/2359112118 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a940cf6a0 con 0x7f1a940ccd70 2023-07-06T01:19:39.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.425+0000 7f1a7bfff700 1 -- 172.21.15.22:0/2359112118 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a88019440 con 0x7f1a940ccd70 2023-07-06T01:19:39.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.425+0000 7f1a7bfff700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1a8006ace0 0x7f1a8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.425+0000 7f1a7bfff700 1 -- 172.21.15.22:0/2359112118 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a8808b1d0 con 0x7f1a940ccd70 2023-07-06T01:19:39.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.425+0000 7f1a937fe700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1a8006ace0 0x7f1a8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.426+0000 7f1a937fe700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1a8006ace0 0x7f1a8006d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1a84000f60 tx=0x7f1a84007c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:39.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.429+0000 7f1a7bfff700 1 -- 172.21.15.22:0/2359112118 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1a940cf6a0 con 0x7f1a940ccd70 2023-07-06T01:19:39.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.669+0000 7f1a99b99700 1 -- 172.21.15.22:0/2359112118 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a940cf6a0 con 0x7f1a940ccd70 2023-07-06T01:19:39.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.670+0000 7f1a7bfff700 1 -- 172.21.15.22:0/2359112118 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4671 (secure 0 0 0) 0x7f1a940cf6a0 con 0x7f1a940ccd70 2023-07-06T01:19:39.673 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:39.673 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/2496681609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2496681609},{"type":"v1","addr":"172.21.15.22:6833","nonce":2496681609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2891594418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2891594418},{"type":"v1","addr":"172.21.15.22:6835","nonce":2891594418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/4097705589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4097705589},{"type":"v1","addr":"172.21.15.103:6836","nonce":4097705589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T01:19:25.042193+0000","modified":"2023-07-06T01:19:28.649724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6837/4015300665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":4015300665},{"type":"v1","addr":"172.21.15.103:6837","nonce":4015300665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T01:19:39.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.672+0000 7f1a99b99700 1 -- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1a8006ace0 msgr2=0x7f1a8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.672+0000 7f1a99b99700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1a8006ace0 0x7f1a8006d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1a84000f60 tx=0x7f1a84007c90 comp rx=0 tx=0).stop 2023-07-06T01:19:39.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.672+0000 7f1a99b99700 1 -- 172.21.15.22:0/2359112118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a940ccd70 msgr2=0x7f1a940c8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.672+0000 7f1a99b99700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a940ccd70 0x7f1a940c8550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1a880106b0 tx=0x7f1a88010a70 comp rx=0 tx=0).stop 2023-07-06T01:19:39.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.672+0000 7f1a99b99700 1 -- 172.21.15.22:0/2359112118 shutdown_connections 2023-07-06T01:19:39.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.672+0000 7f1a99b99700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1a8006ace0 0x7f1a8006d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.672+0000 7f1a99b99700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a94159360 0x7f1a940c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.672+0000 7f1a99b99700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a9406d040 0x7f1a940c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.672+0000 7f1a99b99700 1 --2- 172.21.15.22:0/2359112118 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a940ccd70 0x7f1a940c8550 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.672+0000 7f1a99b99700 1 -- 172.21.15.22:0/2359112118 >> 172.21.15.22:0/2359112118 conn(0x7f1a940b1890 msgr2=0x7f1a940b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:39.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.672+0000 7f1a99b99700 1 -- 172.21.15.22:0/2359112118 shutdown_connections 2023-07-06T01:19:39.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.672+0000 7f1a99b99700 1 -- 172.21.15.22:0/2359112118 wait complete. 2023-07-06T01:19:39.677 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2023-07-06T01:19:39.687 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:39.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.809+0000 7f618603e700 1 Processor -- start 2023-07-06T01:19:39.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.809+0000 7f618603e700 1 -- start start 2023-07-06T01:19:39.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.809+0000 7f618603e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f618015a710 0x7f61800ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.809+0000 7f618603e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61800cef70 0x7f61800cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.809+0000 7f618603e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61800af270 con 0x7f61800aecb0 2023-07-06T01:19:39.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.809+0000 7f618603e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61800af550 con 0x7f618015a710 2023-07-06T01:19:39.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.809+0000 7f618603e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61800af830 con 0x7f61800cef70 2023-07-06T01:19:39.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.810+0000 7f617f7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f618015a710 0x7f61800ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.810+0000 7f617effd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f61800aecb0 0x7f61800ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36724/0 (socket says 172.21.15.22:36724) 2023-07-06T01:19:39.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.810+0000 7f617effd700 1 -- 172.21.15.22:0/3267470956 learned_addr learned my addr 172.21.15.22:0/3267470956 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:39.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.810+0000 7f617f7fe700 1 -- 172.21.15.22:0/3267470956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61800cef70 msgr2=0x7f61800cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:39.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.810+0000 7f617ffff700 1 --2- 172.21.15.22:0/3267470956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61800cef70 0x7f61800cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.810+0000 7f617f7fe700 1 --2- 172.21.15.22:0/3267470956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61800cef70 0x7f61800cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.810+0000 7f617e7fc700 1 -- 172.21.15.22:0/3267470956 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611268507 0 0) 0x7f61800af270 con 0x7f61800aecb0 2023-07-06T01:19:39.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.810+0000 7f617f7fe700 1 -- 172.21.15.22:0/3267470956 >> v1:172.21.15.22:6789/0 conn(0x7f61800aecb0 legacy=0x7f61800ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.810+0000 7f617f7fe700 1 -- 172.21.15.22:0/3267470956 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61800cb2e0 con 0x7f618015a710 2023-07-06T01:19:39.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.810+0000 7f617f7fe700 1 --2- 172.21.15.22:0/3267470956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f618015a710 0x7f61800ae770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6170009840 tx=0x7f6170009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decbc9612ef88fb5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:39.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.810+0000 7f617e7fc700 1 -- 172.21.15.22:0/3267470956 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170011830 con 0x7f618015a710 2023-07-06T01:19:39.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.810+0000 7f617e7fc700 1 -- 172.21.15.22:0/3267470956 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6170011990 con 0x7f618015a710 2023-07-06T01:19:39.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.810+0000 7f617e7fc700 1 -- 172.21.15.22:0/3267470956 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170011d20 con 0x7f618015a710 2023-07-06T01:19:39.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.811+0000 7f618603e700 1 -- 172.21.15.22:0/3267470956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f618015a710 msgr2=0x7f61800ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.811+0000 7f618603e700 1 --2- 172.21.15.22:0/3267470956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f618015a710 0x7f61800ae770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6170009840 tx=0x7f6170009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.811+0000 7f618603e700 1 -- 172.21.15.22:0/3267470956 shutdown_connections 2023-07-06T01:19:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.811+0000 7f618603e700 1 --2- 172.21.15.22:0/3267470956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61800cef70 0x7f61800cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.811+0000 7f618603e700 1 --2- 172.21.15.22:0/3267470956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f618015a710 0x7f61800ae770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.811+0000 7f618603e700 1 -- 172.21.15.22:0/3267470956 >> 172.21.15.22:0/3267470956 conn(0x7f61800999f0 msgr2=0x7f6180099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.811+0000 7f618603e700 1 -- 172.21.15.22:0/3267470956 shutdown_connections 2023-07-06T01:19:39.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.811+0000 7f618603e700 1 -- 172.21.15.22:0/3267470956 wait complete. 2023-07-06T01:19:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.812+0000 7f618603e700 1 Processor -- start 2023-07-06T01:19:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.812+0000 7f618603e700 1 -- start start 2023-07-06T01:19:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f618603e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61800aecb0 0x7f618014e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f618603e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61800cef70 0x7f618006c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f618603e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f618015a710 0x7f61800c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f618603e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618006bec0 con 0x7f61800aecb0 2023-07-06T01:19:39.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f618603e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618006b240 con 0x7f61800cef70 2023-07-06T01:19:39.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f618603e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618006b520 con 0x7f618015a710 2023-07-06T01:19:39.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f617f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61800aecb0 0x7f618014e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f617f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61800aecb0 0x7f618014e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45986/0 (socket says 172.21.15.22:45986) 2023-07-06T01:19:39.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f617f7fe700 1 -- 172.21.15.22:0/2538445906 learned_addr learned my addr 172.21.15.22:0/2538445906 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:39.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f617effd700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61800cef70 0x7f618006c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f617ffff700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f618015a710 0x7f61800c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f617f7fe700 1 -- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f618015a710 msgr2=0x7f61800c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f617f7fe700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f618015a710 0x7f61800c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f617f7fe700 1 -- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61800cef70 msgr2=0x7f618006c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:39.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f617f7fe700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61800cef70 0x7f618006c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:39.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f617f7fe700 1 -- 172.21.15.22:0/2538445906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700094a0 con 0x7f61800aecb0 2023-07-06T01:19:39.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f617ffff700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f618015a710 0x7f61800c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:39.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.813+0000 7f617f7fe700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61800aecb0 0x7f618014e120 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6170014040 tx=0x7f6170004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:39.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.814+0000 7f6167fff700 1 -- 172.21.15.22:0/2538445906 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700036a0 con 0x7f61800aecb0 2023-07-06T01:19:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.814+0000 7f6167fff700 1 -- 172.21.15.22:0/2538445906 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617001e070 con 0x7f61800aecb0 2023-07-06T01:19:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.814+0000 7f618603e700 1 -- 172.21.15.22:0/2538445906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618006b800 con 0x7f61800aecb0 2023-07-06T01:19:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.814+0000 7f6167fff700 1 -- 172.21.15.22:0/2538445906 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170003a20 con 0x7f61800aecb0 2023-07-06T01:19:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.814+0000 7f618603e700 1 -- 172.21.15.22:0/2538445906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618014e660 con 0x7f61800aecb0 2023-07-06T01:19:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.815+0000 7f618603e700 1 -- 172.21.15.22:0/2538445906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618006c130 con 0x7f61800aecb0 2023-07-06T01:19:39.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.815+0000 7f6167fff700 1 -- 172.21.15.22:0/2538445906 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6170003b80 con 0x7f61800aecb0 2023-07-06T01:19:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.815+0000 7f6167fff700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f616c06ace0 0x7f616c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.816+0000 7f6167fff700 1 -- 172.21.15.22:0/2538445906 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f617008a450 con 0x7f61800aecb0 2023-07-06T01:19:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.816+0000 7f617effd700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f616c06ace0 0x7f616c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.816+0000 7f617effd700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f616c06ace0 0x7f616c06d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f61801f5d50 tx=0x7f6174007be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:39.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:39.820+0000 7f6167fff700 1 -- 172.21.15.22:0/2538445906 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6170055590 con 0x7f61800aecb0 2023-07-06T01:19:40.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.019+0000 7f618603e700 1 -- 172.21.15.22:0/2538445906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61800744b0 con 0x7f61800aecb0 2023-07-06T01:19:40.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.020+0000 7f6167fff700 1 -- 172.21.15.22:0/2538445906 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7f617002c3b0 con 0x7f61800aecb0 2023-07-06T01:19:40.023 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:40.024 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:40.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.023+0000 7f618603e700 1 -- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f616c06ace0 msgr2=0x7f616c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:40.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.023+0000 7f618603e700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f616c06ace0 0x7f616c06d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f61801f5d50 tx=0x7f6174007be0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.023+0000 7f618603e700 1 -- 172.21.15.22:0/2538445906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61800aecb0 msgr2=0x7f618014e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:40.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.023+0000 7f618603e700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61800aecb0 0x7f618014e120 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6170014040 tx=0x7f6170004750 comp rx=0 tx=0).stop 2023-07-06T01:19:40.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.023+0000 7f618603e700 1 -- 172.21.15.22:0/2538445906 shutdown_connections 2023-07-06T01:19:40.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.023+0000 7f618603e700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f618015a710 0x7f61800c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.023+0000 7f618603e700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61800aecb0 0x7f618014e120 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.023+0000 7f618603e700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f616c06ace0 0x7f616c06d190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.023+0000 7f618603e700 1 --2- 172.21.15.22:0/2538445906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61800cef70 0x7f618006c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.023+0000 7f618603e700 1 -- 172.21.15.22:0/2538445906 >> 172.21.15.22:0/2538445906 conn(0x7f61800999f0 msgr2=0x7f61800ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:40.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.023+0000 7f618603e700 1 -- 172.21.15.22:0/2538445906 shutdown_connections 2023-07-06T01:19:40.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.023+0000 7f618603e700 1 -- 172.21.15.22:0/2538445906 wait complete. 2023-07-06T01:19:40.038 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:19:40.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.160+0000 7f9d15489700 1 Processor -- start 2023-07-06T01:19:40.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.160+0000 7f9d15489700 1 -- start start 2023-07-06T01:19:40.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d15489700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d100b36f0 0x7f9d100cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d15489700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9d100d11a0 0x7f9d100b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d15489700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d100cebd0 con 0x7f9d100cf5a0 2023-07-06T01:19:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d15489700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1015cff0 con 0x7f9d100d11a0 2023-07-06T01:19:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d15489700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100cdfc0 con 0x7f9d100b36f0 2023-07-06T01:19:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0e7fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9d100cf5a0 0x7f9d100d0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36738/0 (socket says 172.21.15.22:36738) 2023-07-06T01:19:40.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0e7fc700 1 -- 172.21.15.22:0/619136092 learned_addr learned my addr 172.21.15.22:0/619136092 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:40.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0f7fe700 1 --2- 172.21.15.22:0/619136092 >> v2:172.21.15.103:3300/0 conn(0x7f9d100d11a0 0x7f9d100b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0effd700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d100b36f0 0x7f9d100cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143101897 0 0) 0x7f9d100cebd0 con 0x7f9d100cf5a0 2023-07-06T01:19:40.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf80035b0 con 0x7f9d100cf5a0 2023-07-06T01:19:40.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3668545194 0 0) 0x7f9cf80035b0 con 0x7f9d100cf5a0 2023-07-06T01:19:40.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d100b36f0 msgr2=0x7f9d100cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:40.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d100b36f0 0x7f9d100cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 >> v2:172.21.15.103:3300/0 conn(0x7f9d100d11a0 msgr2=0x7f9d100b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 --2- 172.21.15.22:0/619136092 >> v2:172.21.15.103:3300/0 conn(0x7f9d100d11a0 0x7f9d100b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d100ce2a0 con 0x7f9d100cf5a0 2023-07-06T01:19:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f9d04001670 con 0x7f9d100cf5a0 2023-07-06T01:19:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 >> v1:172.21.15.22:6789/0 conn(0x7f9d100cf5a0 legacy=0x7f9d100d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cf8003a00 0x7f9cf8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cf8004430 0x7f9cf80048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d100b36f0 0x7f9cf8005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1015cff0 con 0x7f9cf8004430 2023-07-06T01:19:40.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100cdfc0 con 0x7f9cf8003a00 2023-07-06T01:19:40.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf800ba10 con 0x7f9d100b36f0 2023-07-06T01:19:40.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0f7fe700 1 --2- 172.21.15.22:0/619136092 >> v2:172.21.15.103:3300/0 conn(0x7f9d100d11a0 0x7f9d100b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:40.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.161+0000 7f9d0e7fc700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cf8004430 0x7f9cf80048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.162+0000 7f9d0effd700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cf8003a00 0x7f9cf8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.162+0000 7f9d0f7fe700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d100b36f0 0x7f9cf8005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.162+0000 7f9d0e7fc700 1 -- 172.21.15.22:0/619136092 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d100b36f0 msgr2=0x7f9cf8005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:40.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.162+0000 7f9d0e7fc700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d100b36f0 0x7f9cf8005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.162+0000 7f9d0e7fc700 1 -- 172.21.15.22:0/619136092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cf8003a00 msgr2=0x7f9cf8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:40.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.162+0000 7f9d0e7fc700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cf8003a00 0x7f9cf8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.162+0000 7f9d0e7fc700 1 -- 172.21.15.22:0/619136092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80035b0 con 0x7f9cf8004430 2023-07-06T01:19:40.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.162+0000 7f9d0effd700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9cf8003a00 0x7f9cf8003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:40.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.162+0000 7f9d0f7fe700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9d100b36f0 0x7f9cf8005060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:40.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.162+0000 7f9d0e7fc700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cf8004430 0x7f9cf80048f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400c350 tx=0x7f9cf8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c006fc8e15450082 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:40.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.162+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0400ce50 con 0x7f9cf8004430 2023-07-06T01:19:40.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.162+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d04005960 con 0x7f9cf8004430 2023-07-06T01:19:40.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.162+0000 7f9d0dffb700 1 -- 172.21.15.22:0/619136092 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04005d10 con 0x7f9cf8004430 2023-07-06T01:19:40.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.163+0000 7f9d15489700 1 -- 172.21.15.22:0/619136092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cf8004430 msgr2=0x7f9cf80048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:40.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.163+0000 7f9d15489700 1 --2- 172.21.15.22:0/619136092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9cf8004430 0x7f9cf80048f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400c350 tx=0x7f9cf8005880 comp rx=0 tx=0).stop 2023-07-06T01:19:40.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.163+0000 7f9d0f7fe700 1 -- 172.21.15.22:0/619136092 reap_dead start 2023-07-06T01:19:40.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.163+0000 7f9d15489700 1 -- 172.21.15.22:0/619136092 shutdown_connections 2023-07-06T01:19:40.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.163+0000 7f9d15489700 1 -- 172.21.15.22:0/619136092 >> 172.21.15.22:0/619136092 conn(0x7f9d1009c2e0 msgr2=0x7f9d100d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:40.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.163+0000 7f9d15489700 1 -- 172.21.15.22:0/619136092 shutdown_connections 2023-07-06T01:19:40.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.163+0000 7f9d15489700 1 -- 172.21.15.22:0/619136092 wait complete. 2023-07-06T01:19:40.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.164+0000 7f9d15489700 1 Processor -- start 2023-07-06T01:19:40.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.164+0000 7f9d15489700 1 -- start start 2023-07-06T01:19:40.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d15489700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9cf8004430 0x7f9d101f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d15489700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d1015db40 0x7f9d10157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d15489700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d10157a90 0x7f9d10157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d15489700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100cbb30 con 0x7f9d10157a90 2023-07-06T01:19:40.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d15489700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d101f5560 con 0x7f9d1015db40 2023-07-06T01:19:40.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d15489700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1015e0b0 con 0x7f9cf8004430 2023-07-06T01:19:40.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d0f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d10157a90 0x7f9d10157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d0f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d10157a90 0x7f9d10157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46010/0 (socket says 172.21.15.22:46010) 2023-07-06T01:19:40.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d0f7fe700 1 -- 172.21.15.22:0/3794653201 learned_addr learned my addr 172.21.15.22:0/3794653201 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:40.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d0effd700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9cf8004430 0x7f9d101f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d0e7fc700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d1015db40 0x7f9d10157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d0f7fe700 1 -- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9cf8004430 msgr2=0x7f9d101f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:40.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d0f7fe700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9cf8004430 0x7f9d101f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d0f7fe700 1 -- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d1015db40 msgr2=0x7f9d10157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:40.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d0f7fe700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d1015db40 0x7f9d10157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d0f7fe700 1 -- 172.21.15.22:0/3794653201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1015e390 con 0x7f9d10157a90 2023-07-06T01:19:40.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.165+0000 7f9d0f7fe700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d10157a90 0x7f9d10157f00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9d1006a750 tx=0x7f9d080114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:40.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.166+0000 7f9cf77fe700 1 -- 172.21.15.22:0/3794653201 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800e070 con 0x7f9d10157a90 2023-07-06T01:19:40.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.166+0000 7f9cf77fe700 1 -- 172.21.15.22:0/3794653201 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d08003c30 con 0x7f9d10157a90 2023-07-06T01:19:40.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.166+0000 7f9d15489700 1 -- 172.21.15.22:0/3794653201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1015e670 con 0x7f9d10157a90 2023-07-06T01:19:40.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.166+0000 7f9cf77fe700 1 -- 172.21.15.22:0/3794653201 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d08003de0 con 0x7f9d10157a90 2023-07-06T01:19:40.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.166+0000 7f9d15489700 1 -- 172.21.15.22:0/3794653201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d10158730 con 0x7f9d10157a90 2023-07-06T01:19:40.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.167+0000 7f9d15489700 1 -- 172.21.15.22:0/3794653201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d100b4840 con 0x7f9d10157a90 2023-07-06T01:19:40.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.169+0000 7f9cf77fe700 1 -- 172.21.15.22:0/3794653201 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d08011900 con 0x7f9d10157a90 2023-07-06T01:19:40.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.169+0000 7f9cf77fe700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9cfc06ace0 0x7f9cfc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.169+0000 7f9cf77fe700 1 -- 172.21.15.22:0/3794653201 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9d0808a350 con 0x7f9d10157a90 2023-07-06T01:19:40.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.169+0000 7f9d0effd700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9cfc06ace0 0x7f9cfc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.170+0000 7f9d0effd700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9cfc06ace0 0x7f9cfc06d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009b10 tx=0x7f9d00008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:40.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.173+0000 7f9cf77fe700 1 -- 172.21.15.22:0/3794653201 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9d080554e0 con 0x7f9d10157a90 2023-07-06T01:19:40.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.418+0000 7f9d15489700 1 -- 172.21.15.22:0/3794653201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d100744b0 con 0x7f9d10157a90 2023-07-06T01:19:40.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.419+0000 7f9cf77fe700 1 -- 172.21.15.22:0/3794653201 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4671 (secure 0 0 0) 0x7f9d08058b00 con 0x7f9d10157a90 2023-07-06T01:19:40.422 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:40.422 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/2496681609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2496681609},{"type":"v1","addr":"172.21.15.22:6833","nonce":2496681609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2891594418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2891594418},{"type":"v1","addr":"172.21.15.22:6835","nonce":2891594418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/4097705589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4097705589},{"type":"v1","addr":"172.21.15.103:6836","nonce":4097705589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T01:19:25.042193+0000","modified":"2023-07-06T01:19:28.649724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6837/4015300665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":4015300665},{"type":"v1","addr":"172.21.15.103:6837","nonce":4015300665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T01:19:40.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.421+0000 7f9d15489700 1 -- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9cfc06ace0 msgr2=0x7f9cfc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:40.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.421+0000 7f9d15489700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9cfc06ace0 0x7f9cfc06d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009b10 tx=0x7f9d00008040 comp rx=0 tx=0).stop 2023-07-06T01:19:40.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.421+0000 7f9d15489700 1 -- 172.21.15.22:0/3794653201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d10157a90 msgr2=0x7f9d10157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:40.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.421+0000 7f9d15489700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d10157a90 0x7f9d10157f00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9d1006a750 tx=0x7f9d080114b0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.421+0000 7f9d15489700 1 -- 172.21.15.22:0/3794653201 shutdown_connections 2023-07-06T01:19:40.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.421+0000 7f9d15489700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9d10157a90 0x7f9d10157f00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.421+0000 7f9d15489700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9cf8004430 0x7f9d101f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.421+0000 7f9d15489700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9cfc06ace0 0x7f9cfc06d190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.421+0000 7f9d15489700 1 --2- 172.21.15.22:0/3794653201 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d1015db40 0x7f9d10157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.421+0000 7f9d15489700 1 -- 172.21.15.22:0/3794653201 >> 172.21.15.22:0/3794653201 conn(0x7f9d1009c2e0 msgr2=0x7f9d100d15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:40.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.421+0000 7f9d15489700 1 -- 172.21.15.22:0/3794653201 shutdown_connections 2023-07-06T01:19:40.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.421+0000 7f9d15489700 1 -- 172.21.15.22:0/3794653201 wait complete. 2023-07-06T01:19:40.426 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2023-07-06T01:19:40.839 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T01:19:25.042193+0000', 'modified': '2023-07-06T01:19:28.649724+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.103:6837/4015300665', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6835', 'nonce': 4015300665}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 4015300665}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T01:19:40.839 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T01:19:40.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-07-06T01:19:40.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.966+0000 7f28bbd9e700 1 Processor -- start 2023-07-06T01:19:40.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.966+0000 7f28bbd9e700 1 -- start start 2023-07-06T01:19:40.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.967+0000 7f28bbd9e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f28b4159360 0x7f28b415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.967+0000 7f28bbd9e700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f28b40cc660 0x7f28b40ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.967+0000 7f28bbd9e700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b40cf230 con 0x7f28b4070a20 2023-07-06T01:19:40.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.967+0000 7f28bbd9e700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b415df20 con 0x7f28b40cc660 2023-07-06T01:19:40.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.967+0000 7f28bbd9e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b40ccff0 con 0x7f28b4159360 2023-07-06T01:19:40.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.967+0000 7f28b9b3a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f28b4159360 0x7f28b415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.967+0000 7f28b9339700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f28b40cc660 0x7f28b40ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.967+0000 7f28ba33b700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f28b4070a20 0x7f28b4070e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:38826/0 (socket says 172.21.15.22:38826) 2023-07-06T01:19:40.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.967+0000 7f28ba33b700 1 -- 172.21.15.22:0/1762685537 learned_addr learned my addr 172.21.15.22:0/1762685537 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:40.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.967+0000 7f28b9339700 1 --2- 172.21.15.22:0/1762685537 >> v2:172.21.15.103:3300/0 conn(0x7f28b40cc660 0x7f28b40ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:34740/0 (socket says 172.21.15.22:34740) 2023-07-06T01:19:40.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.967+0000 7f28b9b3a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f28b4159360 0x7f28b415d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:57152/0 (socket says 172.21.15.22:57152) 2023-07-06T01:19:40.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.968+0000 7f28b9339700 1 -- 172.21.15.22:0/1762685537 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f28b4159360 msgr2=0x7f28b415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:40.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.968+0000 7f28b9339700 1 --2- 172.21.15.22:0/1762685537 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f28b4159360 0x7f28b415d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.968+0000 7f28b9339700 1 -- 172.21.15.22:0/1762685537 >> v1:172.21.15.103:6789/0 conn(0x7f28b4070a20 legacy=0x7f28b4070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:40.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.968+0000 7f28b9339700 1 -- 172.21.15.22:0/1762685537 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b40ce620 con 0x7f28b40cc660 2023-07-06T01:19:40.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.968+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/1762685537 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f28b4159360 0x7f28b415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:40.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.968+0000 7f28b9339700 1 --2- 172.21.15.22:0/1762685537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b40cc660 0x7f28b40ccab0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009840 tx=0x7f28b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=660642ca31fe5d49 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:40.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.968+0000 7f28b8b38700 1 -- 172.21.15.22:0/1762685537 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0011830 con 0x7f28b40cc660 2023-07-06T01:19:40.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.968+0000 7f28b8b38700 1 -- 172.21.15.22:0/1762685537 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b0011990 con 0x7f28b40cc660 2023-07-06T01:19:40.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.968+0000 7f28b8b38700 1 -- 172.21.15.22:0/1762685537 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0011d20 con 0x7f28b40cc660 2023-07-06T01:19:40.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.969+0000 7f28bbd9e700 1 -- 172.21.15.22:0/1762685537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b40cc660 msgr2=0x7f28b40ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:40.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.969+0000 7f28bbd9e700 1 --2- 172.21.15.22:0/1762685537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b40cc660 0x7f28b40ccab0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009840 tx=0x7f28b0009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:40.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.969+0000 7f28bbd9e700 1 -- 172.21.15.22:0/1762685537 shutdown_connections 2023-07-06T01:19:40.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.969+0000 7f28bbd9e700 1 --2- 172.21.15.22:0/1762685537 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f28b4159360 0x7f28b415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.969+0000 7f28bbd9e700 1 --2- 172.21.15.22:0/1762685537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b40cc660 0x7f28b40ccab0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.969+0000 7f28bbd9e700 1 -- 172.21.15.22:0/1762685537 >> 172.21.15.22:0/1762685537 conn(0x7f28b40b1890 msgr2=0x7f28b40b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:40.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.969+0000 7f28bbd9e700 1 -- 172.21.15.22:0/1762685537 shutdown_connections 2023-07-06T01:19:40.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.969+0000 7f28bbd9e700 1 -- 172.21.15.22:0/1762685537 wait complete. 2023-07-06T01:19:40.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.970+0000 7f28bbd9e700 1 Processor -- start 2023-07-06T01:19:40.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.970+0000 7f28bbd9e700 1 -- start start 2023-07-06T01:19:40.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28bbd9e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b4070a20 0x7f28b406d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28bbd9e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b40cc660 0x7f28b40c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28bbd9e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f28b4159360 0x7f28b40c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28bbd9e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b40b5a40 con 0x7f28b4070a20 2023-07-06T01:19:40.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28bbd9e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b41420b0 con 0x7f28b40cc660 2023-07-06T01:19:40.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28bbd9e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b40b5480 con 0x7f28b4159360 2023-07-06T01:19:40.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28b9b3a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b4070a20 0x7f28b406d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28b9b3a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b4070a20 0x7f28b406d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46020/0 (socket says 172.21.15.22:46020) 2023-07-06T01:19:40.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28b9b3a700 1 -- 172.21.15.22:0/3965530847 learned_addr learned my addr 172.21.15.22:0/3965530847 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:40.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28b9339700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b40cc660 0x7f28b40c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28ba33b700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f28b4159360 0x7f28b40c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28b9b3a700 1 -- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f28b4159360 msgr2=0x7f28b40c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:40.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f28b4159360 0x7f28b40c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28b9b3a700 1 -- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b40cc660 msgr2=0x7f28b40c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:40.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b40cc660 0x7f28b40c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28b9b3a700 1 -- 172.21.15.22:0/3965530847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a40094a0 con 0x7f28b4070a20 2023-07-06T01:19:40.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28b9339700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b40cc660 0x7f28b40c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:40.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b4070a20 0x7f28b406d880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28a400ebe0 tx=0x7f28a400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:40.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28aa7fc700 1 -- 172.21.15.22:0/3965530847 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a4009760 con 0x7f28b4070a20 2023-07-06T01:19:40.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28aa7fc700 1 -- 172.21.15.22:0/3965530847 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a400c850 con 0x7f28b4070a20 2023-07-06T01:19:40.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.971+0000 7f28bbd9e700 1 -- 172.21.15.22:0/3965530847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b00094a0 con 0x7f28b4070a20 2023-07-06T01:19:40.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.972+0000 7f28aa7fc700 1 -- 172.21.15.22:0/3965530847 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a400cbe0 con 0x7f28b4070a20 2023-07-06T01:19:40.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.972+0000 7f28bbd9e700 1 -- 172.21.15.22:0/3965530847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b4142350 con 0x7f28b4070a20 2023-07-06T01:19:40.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.973+0000 7f28aa7fc700 1 -- 172.21.15.22:0/3965530847 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f28a400cd40 con 0x7f28b4070a20 2023-07-06T01:19:40.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.973+0000 7f28aa7fc700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f289806ace0 0x7f289806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.973+0000 7f28aa7fc700 1 -- 172.21.15.22:0/3965530847 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f28a4089480 con 0x7f28b4070a20 2023-07-06T01:19:40.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.974+0000 7f28b9339700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f289806ace0 0x7f289806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.974+0000 7f28b9339700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f289806ace0 0x7f289806d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f28b0000c00 tx=0x7f28b0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:40.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.975+0000 7f28c7f23b80 1 Processor -- start 2023-07-06T01:19:40.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.975+0000 7f28c7f23b80 1 -- start start 2023-07-06T01:19:40.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.976+0000 7f28c7f23b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56b470 0x55d08d58a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.976+0000 7f28c7f23b80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d541830 0x55d08d56e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.976+0000 7f28c7f23b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d56bcd0 0x55d08d56e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.976+0000 7f28c7f23b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d08d448c60 con 0x55d08d56b470 2023-07-06T01:19:40.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.976+0000 7f28c7f23b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d08d49b9f0 con 0x55d08d56bcd0 2023-07-06T01:19:40.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.976+0000 7f28c7f23b80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d08d494e80 con 0x55d08d541830 2023-07-06T01:19:40.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.976+0000 7f28b9b3a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56b470 0x55d08d58a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.976+0000 7f28b9b3a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56b470 0x55d08d58a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46028/0 (socket says 172.21.15.22:46028) 2023-07-06T01:19:40.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.976+0000 7f28b9b3a700 1 -- 172.21.15.22:0/424836643 learned_addr learned my addr 172.21.15.22:0/424836643 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:40.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.976+0000 7f28ba33b700 1 --2- 172.21.15.22:0/424836643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d56bcd0 0x55d08d56e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.976+0000 7f28b9339700 1 --2- 172.21.15.22:0/424836643 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d541830 0x55d08d56e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.977+0000 7f28b9b3a700 1 -- 172.21.15.22:0/424836643 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d541830 msgr2=0x55d08d56e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:40.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.977+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/424836643 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d541830 0x55d08d56e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.977+0000 7f28b9b3a700 1 -- 172.21.15.22:0/424836643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d56bcd0 msgr2=0x55d08d56e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:40.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.977+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/424836643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d56bcd0 0x55d08d56e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.977+0000 7f28b9b3a700 1 -- 172.21.15.22:0/424836643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d08d2c2d90 con 0x55d08d56b470 2023-07-06T01:19:40.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.977+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/424836643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56b470 0x55d08d58a670 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f28a40877c0 tx=0x7f28a405c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41adbab6e2f5a12 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:40.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.977+0000 7f28977fe700 1 -- 172.21.15.22:0/424836643 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a4056590 con 0x55d08d56b470 2023-07-06T01:19:40.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.977+0000 7f28977fe700 1 -- 172.21.15.22:0/424836643 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a40890c0 con 0x55d08d56b470 2023-07-06T01:19:40.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.977+0000 7f28977fe700 1 -- 172.21.15.22:0/424836643 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a40576e0 con 0x55d08d56b470 2023-07-06T01:19:40.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.978+0000 7f28c7f23b80 1 -- 172.21.15.22:0/424836643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56b470 msgr2=0x55d08d58a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:40.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.978+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/424836643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56b470 0x55d08d58a670 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f28a40877c0 tx=0x7f28a405c1b0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.978+0000 7f28c7f23b80 1 -- 172.21.15.22:0/424836643 shutdown_connections 2023-07-06T01:19:40.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.978+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/424836643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d56bcd0 0x55d08d56e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.978+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/424836643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56b470 0x55d08d58a670 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.978+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/424836643 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d541830 0x55d08d56e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.978+0000 7f28c7f23b80 1 -- 172.21.15.22:0/424836643 >> 172.21.15.22:0/424836643 conn(0x55d08d538e80 msgr2=0x55d08d5891f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:40.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.978+0000 7f28c7f23b80 1 -- 172.21.15.22:0/424836643 shutdown_connections 2023-07-06T01:19:40.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.978+0000 7f28c7f23b80 1 -- 172.21.15.22:0/424836643 wait complete. 2023-07-06T01:19:40.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.978+0000 7f28c7f23b80 1 -- start start 2023-07-06T01:19:40.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.978+0000 7f28c7f23b80 1 Processor -- start 2023-07-06T01:19:40.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.979+0000 7f28c7f23b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d541830 0x55d08d5990c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.979+0000 7f28c7f23b80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 0x55d08d587c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.979+0000 7f28c7f23b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 0x55d08d541c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:40.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.979+0000 7f28c7f23b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d08d434a70 con 0x55d08d56bcd0 2023-07-06T01:19:40.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.979+0000 7f28c7f23b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d08d434770 con 0x55d08d541830 2023-07-06T01:19:40.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.979+0000 7f28c7f23b80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d08d3188d0 con 0x55d08d56b470 2023-07-06T01:19:40.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.979+0000 7f28c7f23b80 10 client.0 _ll_register_callbacks cb 0x55d08d432710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T01:19:40.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.979+0000 7f28ba33b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 0x55d08d541c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.979+0000 7f28ba33b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 0x55d08d541c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46038/0 (socket says 172.21.15.22:46038) 2023-07-06T01:19:40.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.979+0000 7f28ba33b700 1 -- 172.21.15.22:0/4205241324 learned_addr learned my addr 172.21.15.22:0/4205241324 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:40.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28b9339700 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 0x55d08d587c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d541830 0x55d08d5990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:40.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28ba33b700 1 -- 172.21.15.22:0/4205241324 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 msgr2=0x55d08d587c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:40.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28ba33b700 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 0x55d08d587c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:40.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28ba33b700 1 -- 172.21.15.22:0/4205241324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d541830 msgr2=0x55d08d5990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:40.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28ba33b700 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d541830 0x55d08d5990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28ba33b700 1 -- 172.21.15.22:0/4205241324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b4142350 con 0x55d08d56bcd0 2023-07-06T01:19:41.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d541830 0x55d08d5990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:41.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28ba33b700 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 0x55d08d541c60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f28ac012cb0 tx=0x7f28ac012d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a4ab24912dcde9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:41.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f2894ff9700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-07-06T01:19:41.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f2894ff9700 1 -- 172.21.15.22:0/4205241324 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac010940 con 0x55d08d56bcd0 2023-07-06T01:19:41.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f2894ff9700 1 -- 172.21.15.22:0/4205241324 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f28ac010aa0 con 0x55d08d56bcd0 2023-07-06T01:19:41.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28c7f23b80 1 -- 172.21.15.22:0/4205241324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d08d536490 con 0x55d08d56bcd0 2023-07-06T01:19:41.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f2894ff9700 1 -- 172.21.15.22:0/4205241324 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28ac010ca0 con 0x55d08d56bcd0 2023-07-06T01:19:41.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28c7f23b80 10 client.4635 fetch_fsmap learned FSMap version 7 2023-07-06T01:19:41.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28c7f23b80 1 -- 172.21.15.22:0/4205241324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55d08d2abc10 con 0x55d08d56bcd0 2023-07-06T01:19:41.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f2894ff9700 1 -- 172.21.15.22:0/4205241324 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d08d536490 con 0x55d08d56bcd0 2023-07-06T01:19:41.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f2894ff9700 1 -- 172.21.15.22:0/4205241324 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f28ac015440 con 0x55d08d56bcd0 2023-07-06T01:19:41.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28c7f23b80 10 client.4635 fetch_fsmap finished waiting for FSMap version 7 2023-07-06T01:19:41.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28c7f23b80 10 client.4635 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T01:19:41.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28c7f23b80 20 client.4635 populate_metadata read hostname 'smithi022' 2023-07-06T01:19:41.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.980+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.103:6835/4015300665,v1:172.21.15.103:6837/4015300665] conn(0x55d08d5421a0 0x55d08d59c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.981+0000 7f28c7f23b80 4 client.4635 mds_command: new command op to 4288 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T01:19:41.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.981+0000 7f28c7f23b80 1 -- 172.21.15.22:0/4205241324 --> [v2:172.21.15.103:6835/4015300665,v1:172.21.15.103:6837/4015300665] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d08d48de60 con 0x55d08d5421a0 2023-07-06T01:19:41.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.981+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.103:6835/4015300665,v1:172.21.15.103:6837/4015300665] conn(0x55d08d5421a0 0x55d08d59c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.981+0000 7f2894ff9700 10 client.4635 ms_handle_connect on v2:172.21.15.103:6835/4015300665 2023-07-06T01:19:41.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.981+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.103:6835/4015300665,v1:172.21.15.103:6837/4015300665] conn(0x55d08d5421a0 0x55d08d59c1f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e55c0424e2bb9d73 server_cookie=9b0a5b281da67dc9 in_seq=0 out_seq=0 2023-07-06T01:19:41.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f2894ff9700 1 -- 172.21.15.22:0/4205241324 <== mds.0 v2:172.21.15.103:6835/4015300665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d08d48de60 con 0x55d08d5421a0 2023-07-06T01:19:41.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f2894ff9700 10 client.4635 handle_command_reply: tid=0 2023-07-06T01:19:41.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f28c7f23b80 1 client.4635 shutdown 2023-07-06T01:19:41.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f28c7f23b80 1 -- 172.21.15.22:0/4205241324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 msgr2=0x55d08d541c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:41.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 0x55d08d541c60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f28ac012cb0 tx=0x7f28ac012d90 comp rx=0 tx=0).stop 2023-07-06T01:19:41.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f28c7f23b80 1 -- 172.21.15.22:0/4205241324 shutdown_connections 2023-07-06T01:19:41.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.103:6835/4015300665,v1:172.21.15.103:6837/4015300665] conn(0x55d08d5421a0 0x55d08d59c1f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 0x55d08d541c60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d541830 0x55d08d5990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f2894ff9700 0 client.4635 ms_handle_reset on v2:172.21.15.103:6835/4015300665 2023-07-06T01:19:41.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/4205241324 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 0x55d08d587c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f28c7f23b80 1 -- 172.21.15.22:0/4205241324 >> 172.21.15.22:0/4205241324 conn(0x55d08d538e80 msgr2=0x55d08d5681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:41.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f28c7f23b80 1 -- 172.21.15.22:0/4205241324 shutdown_connections 2023-07-06T01:19:41.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f28c7f23b80 1 -- 172.21.15.22:0/4205241324 wait complete. 2023-07-06T01:19:41.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:40.982+0000 7f28c7f23b80 20 client.4635 trim_cache size 0 max 16384 2023-07-06T01:19:41.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.000+0000 7f28c7f23b80 1 Processor -- start 2023-07-06T01:19:41.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.000+0000 7f28c7f23b80 1 -- start start 2023-07-06T01:19:41.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28c7f23b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d541830 0x55d08d582ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28c7f23b80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 0x55d08d5830e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28c7f23b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 0x55d08d599e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28c7f23b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d08d541c00 con 0x55d08d56bcd0 2023-07-06T01:19:41.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28c7f23b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d08d571fc0 con 0x55d08d541830 2023-07-06T01:19:41.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28c7f23b80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d08d535fd0 con 0x55d08d56b470 2023-07-06T01:19:41.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28ba33b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 0x55d08d599e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28ba33b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 0x55d08d599e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46046/0 (socket says 172.21.15.22:46046) 2023-07-06T01:19:41.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28ba33b700 1 -- 172.21.15.22:0/3186963525 learned_addr learned my addr 172.21.15.22:0/3186963525 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:41.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/3186963525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d541830 0x55d08d582ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28b9339700 1 --2- 172.21.15.22:0/3186963525 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 0x55d08d5830e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28ba33b700 1 -- 172.21.15.22:0/3186963525 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 msgr2=0x55d08d5830e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:41.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28ba33b700 1 --2- 172.21.15.22:0/3186963525 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 0x55d08d5830e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28ba33b700 1 -- 172.21.15.22:0/3186963525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d541830 msgr2=0x55d08d582ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:41.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28ba33b700 1 --2- 172.21.15.22:0/3186963525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d541830 0x55d08d582ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28ba33b700 1 -- 172.21.15.22:0/3186963525 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d08d2e9200 con 0x55d08d56bcd0 2023-07-06T01:19:41.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28b9339700 1 --2- 172.21.15.22:0/3186963525 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 0x55d08d5830e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:41.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.001+0000 7f28ba33b700 1 --2- 172.21.15.22:0/3186963525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 0x55d08d599e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005c90 tx=0x7f28ac00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b712925aec45a848 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:41.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.002+0000 7f2897fff700 1 -- 172.21.15.22:0/3186963525 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac015db0 con 0x55d08d56bcd0 2023-07-06T01:19:41.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.002+0000 7f2897fff700 1 -- 172.21.15.22:0/3186963525 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac010940 con 0x55d08d56bcd0 2023-07-06T01:19:41.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.002+0000 7f2897fff700 1 -- 172.21.15.22:0/3186963525 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac010cf0 con 0x55d08d56bcd0 2023-07-06T01:19:41.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.002+0000 7f28c7f23b80 1 -- 172.21.15.22:0/3186963525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 msgr2=0x55d08d599e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:41.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.002+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/3186963525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 0x55d08d599e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005c90 tx=0x7f28ac00c000 comp rx=0 tx=0).stop 2023-07-06T01:19:41.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.002+0000 7f28c7f23b80 1 -- 172.21.15.22:0/3186963525 shutdown_connections 2023-07-06T01:19:41.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.002+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/3186963525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d56bcd0 0x55d08d599e30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.002+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/3186963525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d541830 0x55d08d582ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.002+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/3186963525 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 0x55d08d5830e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.002+0000 7f28c7f23b80 1 -- 172.21.15.22:0/3186963525 >> 172.21.15.22:0/3186963525 conn(0x55d08d538e80 msgr2=0x55d08d5a2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:41.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.002+0000 7f28c7f23b80 1 -- 172.21.15.22:0/3186963525 shutdown_connections 2023-07-06T01:19:41.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.002+0000 7f28c7f23b80 1 -- 172.21.15.22:0/3186963525 wait complete. 2023-07-06T01:19:41.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.003+0000 7f28c7f23b80 1 -- start start 2023-07-06T01:19:41.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.003+0000 7f28c7f23b80 1 Processor -- start 2023-07-06T01:19:41.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28c7f23b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d541830 0x55d08d5eb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28c7f23b80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 0x55d08d5f5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28c7f23b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d56bcd0 0x55d08d59a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28c7f23b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d08d5711f0 con 0x55d08d541830 2023-07-06T01:19:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28c7f23b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d08d5723d0 con 0x55d08d56bcd0 2023-07-06T01:19:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28c7f23b80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d08d5b0a40 con 0x55d08d56b470 2023-07-06T01:19:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28c7f23b80 10 client.0 _ll_register_callbacks cb 0x7f28ac012660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T01:19:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28b9b3a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d541830 0x55d08d5eb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28b9b3a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d541830 0x55d08d5eb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46048/0 (socket says 172.21.15.22:46048) 2023-07-06T01:19:41.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28b9b3a700 1 -- 172.21.15.22:0/1856774214 learned_addr learned my addr 172.21.15.22:0/1856774214 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:41.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28b9339700 1 --2- 172.21.15.22:0/1856774214 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 0x55d08d5f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28ba33b700 1 --2- 172.21.15.22:0/1856774214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d56bcd0 0x55d08d59a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28b9b3a700 1 -- 172.21.15.22:0/1856774214 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 msgr2=0x55d08d5f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:41.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/1856774214 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 0x55d08d5f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28b9b3a700 1 -- 172.21.15.22:0/1856774214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d56bcd0 msgr2=0x55d08d59a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:41.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/1856774214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d56bcd0 0x55d08d59a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28b9b3a700 1 -- 172.21.15.22:0/1856774214 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d08d2abc10 con 0x55d08d541830 2023-07-06T01:19:41.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28b9b3a700 1 --2- 172.21.15.22:0/1856774214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d541830 0x55d08d5eb2e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f28a405c1f0 tx=0x7f28a405e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c763482970a43a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:41.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f2894ff9700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-07-06T01:19:41.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f2894ff9700 1 -- 172.21.15.22:0/1856774214 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a4056590 con 0x55d08d541830 2023-07-06T01:19:41.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f2894ff9700 1 -- 172.21.15.22:0/1856774214 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f28a4086040 con 0x55d08d541830 2023-07-06T01:19:41.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28c7f23b80 1 -- 172.21.15.22:0/1856774214 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28a40890c0 con 0x55d08d541830 2023-07-06T01:19:41.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f2894ff9700 1 -- 172.21.15.22:0/1856774214 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28a40576e0 con 0x55d08d541830 2023-07-06T01:19:41.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.004+0000 7f28c7f23b80 10 client.4641 fetch_fsmap learned FSMap version 7 2023-07-06T01:19:41.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f28c7f23b80 1 -- 172.21.15.22:0/1856774214 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55d08d2c1260 con 0x55d08d541830 2023-07-06T01:19:41.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f2894ff9700 1 -- 172.21.15.22:0/1856774214 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a40890c0 con 0x55d08d541830 2023-07-06T01:19:41.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f2894ff9700 1 -- 172.21.15.22:0/1856774214 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f28a40600b0 con 0x55d08d541830 2023-07-06T01:19:41.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f28c7f23b80 10 client.4641 fetch_fsmap finished waiting for FSMap version 7 2023-07-06T01:19:41.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f28c7f23b80 10 client.4641 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T01:19:41.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f28c7f23b80 20 client.4641 populate_metadata read hostname 'smithi022' 2023-07-06T01:19:41.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/1856774214 >> [v2:172.21.15.103:6835/4015300665,v1:172.21.15.103:6837/4015300665] conn(0x55d08d582ba0 0x55d08d582f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f28c7f23b80 4 client.4641 mds_command: new command op to 4288 tid=0[{"prefix": "status"}] 2023-07-06T01:19:41.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f28c7f23b80 1 -- 172.21.15.22:0/1856774214 --> [v2:172.21.15.103:6835/4015300665,v1:172.21.15.103:6837/4015300665] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d08d450000 con 0x55d08d582ba0 2023-07-06T01:19:41.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f28b9339700 1 --2- 172.21.15.22:0/1856774214 >> [v2:172.21.15.103:6835/4015300665,v1:172.21.15.103:6837/4015300665] conn(0x55d08d582ba0 0x55d08d582f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f2894ff9700 10 client.4641 ms_handle_connect on v2:172.21.15.103:6835/4015300665 2023-07-06T01:19:41.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f28b9339700 1 --2- 172.21.15.22:0/1856774214 >> [v2:172.21.15.103:6835/4015300665,v1:172.21.15.103:6837/4015300665] conn(0x55d08d582ba0 0x55d08d582f70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60c85c7c81401d02 server_cookie=42a316d6b7d5a874 in_seq=0 out_seq=0 2023-07-06T01:19:41.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f2894ff9700 1 -- 172.21.15.22:0/1856774214 <== mds.0 v2:172.21.15.103:6835/4015300665 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55d08d450000 con 0x55d08d582ba0 2023-07-06T01:19:41.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f2894ff9700 10 client.4641 handle_command_reply: tid=0 2023-07-06T01:19:41.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.005+0000 7f28c7f23b80 1 client.4641 shutdown 2023-07-06T01:19:41.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28c7f23b80 1 -- 172.21.15.22:0/1856774214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d541830 msgr2=0x55d08d5eb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:41.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/1856774214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d541830 0x55d08d5eb2e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f28a405c1f0 tx=0x7f28a405e070 comp rx=0 tx=0).stop 2023-07-06T01:19:41.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28c7f23b80 1 -- 172.21.15.22:0/1856774214 shutdown_connections 2023-07-06T01:19:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/1856774214 >> [v2:172.21.15.103:6835/4015300665,v1:172.21.15.103:6837/4015300665] conn(0x55d08d582ba0 0x55d08d582f70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/1856774214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d08d56bcd0 0x55d08d59a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/1856774214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d08d541830 0x55d08d5eb2e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f2894ff9700 0 client.4641 ms_handle_reset on v2:172.21.15.103:6835/4015300665 2023-07-06T01:19:41.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28c7f23b80 1 --2- 172.21.15.22:0/1856774214 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55d08d56b470 0x55d08d5f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28c7f23b80 1 -- 172.21.15.22:0/1856774214 >> 172.21.15.22:0/1856774214 conn(0x55d08d538e80 msgr2=0x55d08d567ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:41.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28c7f23b80 1 -- 172.21.15.22:0/1856774214 shutdown_connections 2023-07-06T01:19:41.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28c7f23b80 1 -- 172.21.15.22:0/1856774214 wait complete. 2023-07-06T01:19:41.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28c7f23b80 20 client.4641 trim_cache size 0 max 16384 2023-07-06T01:19:41.027 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:19:41.027 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "63a894dd-9157-4ce0-adf3-c1bdfe16ec64", 2023-07-06T01:19:41.028 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2023-07-06T01:19:41.028 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4288, 2023-07-06T01:19:41.028 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-07-06T01:19:41.028 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-07-06T01:19:41.029 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-07-06T01:19:41.029 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 15.379068775, 2023-07-06T01:19:41.029 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 7, 2023-07-06T01:19:41.029 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 21, 2023-07-06T01:19:41.029 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 21, 2023-07-06T01:19:41.029 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 21.306090107999999 2023-07-06T01:19:41.029 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:19:41.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28bbd9e700 1 -- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f289806ace0 msgr2=0x7f289806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:41.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28bbd9e700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f289806ace0 0x7f289806d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f28b0000c00 tx=0x7f28b0008030 comp rx=0 tx=0).stop 2023-07-06T01:19:41.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28bbd9e700 1 -- 172.21.15.22:0/3965530847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b4070a20 msgr2=0x7f28b406d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:41.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28bbd9e700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b4070a20 0x7f28b406d880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28a400ebe0 tx=0x7f28a400eef0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28bbd9e700 1 -- 172.21.15.22:0/3965530847 shutdown_connections 2023-07-06T01:19:41.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28bbd9e700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f28b4159360 0x7f28b40c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28bbd9e700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28b4070a20 0x7f28b406d880 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28bbd9e700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f289806ace0 0x7f289806d190 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28bbd9e700 1 --2- 172.21.15.22:0/3965530847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28b40cc660 0x7f28b40c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28bbd9e700 1 -- 172.21.15.22:0/3965530847 >> 172.21.15.22:0/3965530847 conn(0x7f28b40b1890 msgr2=0x7f28b4158590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28bbd9e700 1 -- 172.21.15.22:0/3965530847 shutdown_connections 2023-07-06T01:19:41.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.006+0000 7f28bbd9e700 1 -- 172.21.15.22:0/3965530847 wait complete. 2023-07-06T01:19:41.033 INFO:teuthology.run_tasks:Running task kclient... 2023-07-06T01:19:41.060 INFO:tasks.kclient:Mounting kernel clients... 2023-07-06T01:19:41.060 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-07-06T01:19:41.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:41.061 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-07-06T01:19:41.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:41.079 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2023-07-06T01:19:41.147 INFO:teuthology.orchestra.run.smithi022.stderr:Cannot find device "ceph-brx" 2023-07-06T01:19:41.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:19:41.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:41.150 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-07-06T01:19:41.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:41.207 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2023-07-06T01:19:41.274 INFO:teuthology.orchestra.run.smithi022.stderr:Cannot find device "ceph-brx" 2023-07-06T01:19:41.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:19:41.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:41.276 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2023-07-06T01:19:41.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:41.298 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2023-07-06T01:19:41.372 INFO:teuthology.orchestra.run.smithi103.stderr:Cannot find device "ceph-brx" 2023-07-06T01:19:41.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:19:41.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:41.374 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2023-07-06T01:19:41.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:41.433 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2023-07-06T01:19:41.505 INFO:teuthology.orchestra.run.smithi103.stderr:Cannot find device "ceph-brx" 2023-07-06T01:19:41.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:19:41.506 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-07-06T01:19:41.507 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T01:19:41.507 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T01:19:41.508 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T01:19:41.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.632+0000 7ff51c46c700 1 Processor -- start 2023-07-06T01:19:41.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.632+0000 7ff51c46c700 1 -- start start 2023-07-06T01:19:41.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.633+0000 7ff51c46c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff514159870 0x7ff51415df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.633+0000 7ff51c46c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5140d0300 0x7ff5140ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.633+0000 7ff51c46c700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5140ce550 con 0x7ff5140715f0 2023-07-06T01:19:41.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.633+0000 7ff51c46c700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5140cdb60 con 0x7ff514159870 2023-07-06T01:19:41.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.633+0000 7ff51c46c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51415d490 con 0x7ff5140d0300 2023-07-06T01:19:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.633+0000 7ff51a208700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff514159870 0x7ff51415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.633+0000 7ff51a208700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff514159870 0x7ff51415df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46056/0 (socket says 172.21.15.22:46056) 2023-07-06T01:19:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.633+0000 7ff51a208700 1 -- 172.21.15.22:0/904600639 learned_addr learned my addr 172.21.15.22:0/904600639 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.633+0000 7ff51aa09700 1 --2- 172.21.15.22:0/904600639 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5140d0300 0x7ff5140ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.633+0000 7ff51a208700 1 -- 172.21.15.22:0/904600639 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5140d0300 msgr2=0x7ff5140ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.633+0000 7ff51a208700 1 --2- 172.21.15.22:0/904600639 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5140d0300 0x7ff5140ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.633+0000 7ff51a208700 1 -- 172.21.15.22:0/904600639 >> v1:172.21.15.103:6789/0 conn(0x7ff5140715f0 legacy=0x7ff5140cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.633+0000 7ff51a208700 1 -- 172.21.15.22:0/904600639 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51415d770 con 0x7ff514159870 2023-07-06T01:19:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.634+0000 7ff51a208700 1 --2- 172.21.15.22:0/904600639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff514159870 0x7ff51415df70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff5040098f0 tx=0x7ff504009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531bb97c35efaf56 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.634+0000 7ff519206700 1 -- 172.21.15.22:0/904600639 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504011830 con 0x7ff514159870 2023-07-06T01:19:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.634+0000 7ff519206700 1 -- 172.21.15.22:0/904600639 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff504011990 con 0x7ff514159870 2023-07-06T01:19:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.634+0000 7ff519206700 1 -- 172.21.15.22:0/904600639 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504011d20 con 0x7ff514159870 2023-07-06T01:19:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.634+0000 7ff51c46c700 1 -- 172.21.15.22:0/904600639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff514159870 msgr2=0x7ff51415df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.634+0000 7ff51c46c700 1 --2- 172.21.15.22:0/904600639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff514159870 0x7ff51415df70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff5040098f0 tx=0x7ff504009c00 comp rx=0 tx=0).stop 2023-07-06T01:19:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.634+0000 7ff51c46c700 1 -- 172.21.15.22:0/904600639 shutdown_connections 2023-07-06T01:19:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.634+0000 7ff51c46c700 1 --2- 172.21.15.22:0/904600639 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5140d0300 0x7ff5140ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.634+0000 7ff51c46c700 1 --2- 172.21.15.22:0/904600639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff514159870 0x7ff51415df70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.634+0000 7ff51c46c700 1 -- 172.21.15.22:0/904600639 >> 172.21.15.22:0/904600639 conn(0x7ff51406b110 msgr2=0x7ff51406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.634+0000 7ff51c46c700 1 -- 172.21.15.22:0/904600639 shutdown_connections 2023-07-06T01:19:41.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.634+0000 7ff51c46c700 1 -- 172.21.15.22:0/904600639 wait complete. 2023-07-06T01:19:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.636+0000 7ff51c46c700 1 Processor -- start 2023-07-06T01:19:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.636+0000 7ff51c46c700 1 -- start start 2023-07-06T01:19:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.636+0000 7ff51c46c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5140715f0 0x7ff5141e48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.636+0000 7ff51c46c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5140d0300 0x7ff5140b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff51c46c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff514159870 0x7ff5140b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff51c46c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5141e5c40 con 0x7ff5140d0300 2023-07-06T01:19:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff51c46c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5141e4fc0 con 0x7ff5140715f0 2023-07-06T01:19:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff51c46c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5141e52a0 con 0x7ff514159870 2023-07-06T01:19:41.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff519a07700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5140d0300 0x7ff5140b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff519a07700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5140d0300 0x7ff5140b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46064/0 (socket says 172.21.15.22:46064) 2023-07-06T01:19:41.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff519a07700 1 -- 172.21.15.22:0/1442985157 learned_addr learned my addr 172.21.15.22:0/1442985157 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:41.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff51a208700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5140715f0 0x7ff5141e48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff51aa09700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff514159870 0x7ff5140b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff519a07700 1 -- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff514159870 msgr2=0x7ff5140b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:41.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff519a07700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff514159870 0x7ff5140b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff519a07700 1 -- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5140715f0 msgr2=0x7ff5141e48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:41.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff519a07700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5140715f0 0x7ff5141e48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff519a07700 1 -- 172.21.15.22:0/1442985157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff504009550 con 0x7ff5140d0300 2023-07-06T01:19:41.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff519a07700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5140d0300 0x7ff5140b2ab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c4e0 tx=0x7ff51000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:41.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff50affd700 1 -- 172.21.15.22:0/1442985157 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100052d0 con 0x7ff5140d0300 2023-07-06T01:19:41.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff50affd700 1 -- 172.21.15.22:0/1442985157 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff510005430 con 0x7ff5140d0300 2023-07-06T01:19:41.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff51c46c700 1 -- 172.21.15.22:0/1442985157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5141e5580 con 0x7ff5140d0300 2023-07-06T01:19:41.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff50affd700 1 -- 172.21.15.22:0/1442985157 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100038c0 con 0x7ff5140d0300 2023-07-06T01:19:41.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.637+0000 7ff51c46c700 1 -- 172.21.15.22:0/1442985157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5141f2fe0 con 0x7ff5140d0300 2023-07-06T01:19:41.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.638+0000 7ff51c46c700 1 -- 172.21.15.22:0/1442985157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5141e5d80 con 0x7ff5140d0300 2023-07-06T01:19:41.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.639+0000 7ff50affd700 1 -- 172.21.15.22:0/1442985157 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff510003a20 con 0x7ff5140d0300 2023-07-06T01:19:41.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.639+0000 7ff50affd700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff4f806ace0 0x7ff4f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:41.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.639+0000 7ff50affd700 1 -- 172.21.15.22:0/1442985157 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff510088850 con 0x7ff5140d0300 2023-07-06T01:19:41.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.639+0000 7ff51a208700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff4f806ace0 0x7ff4f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:41.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.640+0000 7ff51a208700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff4f806ace0 0x7ff4f806d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff504014040 tx=0x7ff504008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:41.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.644+0000 7ff50affd700 1 -- 172.21.15.22:0/1442985157 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff510053900 con 0x7ff5140d0300 2023-07-06T01:19:41.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.886+0000 7ff51c46c700 1 -- 172.21.15.22:0/1442985157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff5140744b0 con 0x7ff5140d0300 2023-07-06T01:19:41.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.886+0000 7ff50affd700 1 -- 172.21.15.22:0/1442985157 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff510017060 con 0x7ff5140d0300 2023-07-06T01:19:41.889 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T01:19:41.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.888+0000 7ff51c46c700 1 -- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff4f806ace0 msgr2=0x7ff4f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:41.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.888+0000 7ff51c46c700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff4f806ace0 0x7ff4f806d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff504014040 tx=0x7ff504008030 comp rx=0 tx=0).stop 2023-07-06T01:19:41.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.888+0000 7ff51c46c700 1 -- 172.21.15.22:0/1442985157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5140d0300 msgr2=0x7ff5140b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:41.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.888+0000 7ff51c46c700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5140d0300 0x7ff5140b2ab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c4e0 tx=0x7ff51000c7f0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.888+0000 7ff51c46c700 1 -- 172.21.15.22:0/1442985157 shutdown_connections 2023-07-06T01:19:41.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.888+0000 7ff51c46c700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff514159870 0x7ff5140b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.888+0000 7ff51c46c700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff4f806ace0 0x7ff4f806d190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.888+0000 7ff51c46c700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5140715f0 0x7ff5141e48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.888+0000 7ff51c46c700 1 --2- 172.21.15.22:0/1442985157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5140d0300 0x7ff5140b2ab0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:41.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.888+0000 7ff51c46c700 1 -- 172.21.15.22:0/1442985157 >> 172.21.15.22:0/1442985157 conn(0x7ff51406b110 msgr2=0x7ff5141549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:41.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.888+0000 7ff51c46c700 1 -- 172.21.15.22:0/1442985157 shutdown_connections 2023-07-06T01:19:41.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:41.888+0000 7ff51c46c700 1 -- 172.21.15.22:0/1442985157 wait complete. 2023-07-06T01:19:41.903 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T01:19:41.904 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T01:19:41.904 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2023-07-06T01:19:41.904 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T01:19:41.904 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T01:19:41.904 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T01:19:41.905 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T01:19:41.905 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T01:19:41.905 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T01:19:41.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:41.906 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2023-07-06T01:19:41.923 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T01:19:41.923 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T01:19:41.923 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T01:19:41.924 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:41.924 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-07-06T01:19:41.924 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:41.924 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:19:41.924 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:41.924 INFO:teuthology.orchestra.run.smithi022.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T01:19:41.925 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:41.925 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-06T01:19:41.925 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42279sec preferred_lft 42279sec 2023-07-06T01:19:41.925 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2023-07-06T01:19:41.925 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:41.925 INFO:teuthology.orchestra.run.smithi022.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:19:41.925 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:41.926 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:19:41.926 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:41.926 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-06T01:19:41.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:41.927 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-07-06T01:19:41.927 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-07-06T01:19:41.927 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2023-07-06T01:19:41.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2023-07-06T01:19:41.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2023-07-06T01:19:41.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-06T01:19:41.928 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-07-06T01:19:42.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:42.097 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T01:19:42.125 INFO:teuthology.orchestra.run.smithi022.stdout:1 2023-07-06T01:19:42.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:42.127 DEBUG:teuthology.orchestra.run.smithi022:> route 2023-07-06T01:19:42.195 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2023-07-06T01:19:42.195 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T01:19:42.195 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-06T01:19:42.195 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-06T01:19:42.195 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T01:19:42.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:42.196 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-07-06T01:19:42.197 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-07-06T01:19:42.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-06T01:19:42.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-06T01:19:42.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-06T01:19:42.197 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-07-06T01:19:42.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:42.456 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-07-06T01:19:42.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:42.474 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2023-07-06T01:19:42.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:42.531 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-07-06T01:19:42.532 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-07-06T01:19:42.532 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T01:19:42.532 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-06T01:19:42.532 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-07-06T01:19:42.628 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-06T01:19:42.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:42.629 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-07-06T01:19:42.629 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-07-06T01:19:42.629 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-06T01:19:42.629 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-06T01:19:42.629 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-06T01:19:42.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-06T01:19:42.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-06T01:19:42.630 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-07-06T01:19:42.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:42.840 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-07-06T01:19:42.840 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-07-06T01:19:42.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2023-07-06T01:19:42.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2023-07-06T01:19:42.841 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-07-06T01:19:42.948 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T01:19:42.948 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T01:19:42.968 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T01:19:42.969 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T01:19:42.969 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T01:19:43.024 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-07-06T01:19:43.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:43.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:43.136 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:43.136 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:19:43.264 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:43.264 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:19:43.264 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.103:3300,172.21.15.103:3301" 2023-07-06T01:19:43.265 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with new device syntax: 0@63a894dd-9157-4ce0-adf3-c1bdfe16ec64.cephfs=/ 2023-07-06T01:19:43.265 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.22:3300/172.21.15.103:3300/172.21.15.103:3301" will pass to kernel 2023-07-06T01:19:43.265 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: v2 mount support check returned 2 2023-07-06T01:19:43.265 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-06T01:19:43.265 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T01:19:43.338 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:43.338 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:19:43.338 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.103:3300,172.21.15.103:3301" 2023-07-06T01:19:43.339 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with new device syntax: 0@63a894dd-9157-4ce0-adf3-c1bdfe16ec64.cephfs=/ 2023-07-06T01:19:43.339 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.22:3300/172.21.15.103:3300/172.21.15.103:3301" will pass to kernel 2023-07-06T01:19:43.339 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: v2 mount support check returned 2 2023-07-06T01:19:43.339 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-06T01:19:43.339 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T01:19:43.339 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.103:6789,172.21.15.103:6790" 2023-07-06T01:19:43.339 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.103:6789,172.21.15.103:6790:/ 2023-07-06T01:19:43.340 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=63a894dd-9157-4ce0-adf3-c1bdfe16ec64" will pass to kernel 2023-07-06T01:19:43.341 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T01:19:43.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:43.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T01:19:43.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:43.369 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T01:19:43.458 INFO:teuthology.orchestra.run.smithi022.stdout:client4496 2023-07-06T01:19:43.459 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:43.459 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:19:43.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:43.476 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T01:19:43.565 INFO:teuthology.orchestra.run.smithi022.stdout:client4496 2023-07-06T01:19:43.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:43.566 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T01:19:43.614 INFO:teuthology.orchestra.run.smithi022.stdout:client4496 2023-07-06T01:19:43.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:43.615 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/63a894dd-9157-4ce0-adf3-c1bdfe16ec64.client4496/status') 2023-07-06T01:19:43.669 INFO:teuthology.orchestra.run.smithi022.stdout:instance: client.4496 (3)172.21.15.22:0/3740019298 2023-07-06T01:19:43.669 INFO:teuthology.orchestra.run.smithi022.stdout:blocklisted: false 2023-07-06T01:19:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records in 2023-07-06T01:19:43.670 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records out 2023-07-06T01:19:43.671 INFO:teuthology.orchestra.run.smithi022.stderr:70 bytes copied, 2.3233e-05 s, 3.0 MB/s 2023-07-06T01:19:43.671 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-07-06T01:19:43.671 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T01:19:43.672 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-06T01:19:43.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T01:19:43.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.794+0000 7fdfeb9ef700 1 Processor -- start 2023-07-06T01:19:43.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.794+0000 7fdfeb9ef700 1 -- start start 2023-07-06T01:19:43.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.795+0000 7fdfeb9ef700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdfe406a710 0x7fdfe406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:43.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.795+0000 7fdfeb9ef700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe40af670 con 0x7fdfe40af2a0 2023-07-06T01:19:43.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.795+0000 7fdfeb9ef700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe406d030 con 0x7fdfe40cbc10 2023-07-06T01:19:43.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.795+0000 7fdfeb9ef700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40cd560 con 0x7fdfe406a710 2023-07-06T01:19:43.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.795+0000 7fdfe978b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fdfe40af2a0 0x7fdfe406a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36832/0 (socket says 172.21.15.22:36832) 2023-07-06T01:19:43.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.795+0000 7fdfe978b700 1 -- 172.21.15.22:0/779330558 learned_addr learned my addr 172.21.15.22:0/779330558 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:43.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.795+0000 7fdfe8f8a700 1 --2- 172.21.15.22:0/779330558 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdfe406a710 0x7fdfe406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:43.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485646024 0 0) 0x7fdfe40af670 con 0x7fdfe40af2a0 2023-07-06T01:19:43.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd00035b0 con 0x7fdfe40af2a0 2023-07-06T01:19:43.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2775173675 0 0) 0x7fdfd00035b0 con 0x7fdfe40af2a0 2023-07-06T01:19:43.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdfe406a710 msgr2=0x7fdfe406caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:43.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 --2- 172.21.15.22:0/779330558 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdfe406a710 0x7fdfe406caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:43.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 >> v1:172.21.15.103:6789/0 conn(0x7fdfe40cbc10 legacy=0x7fdfe40cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:43.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe40cc760 con 0x7fdfe40af2a0 2023-07-06T01:19:43.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7fdfd4001670 con 0x7fdfe40af2a0 2023-07-06T01:19:43.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 >> v1:172.21.15.22:6789/0 conn(0x7fdfe40af2a0 legacy=0x7fdfe406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:43.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 --2- 172.21.15.22:0/779330558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd0003a30 0x7fdfd0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:43.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 --2- 172.21.15.22:0/779330558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfd00043d0 0x7fdfd0004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:43.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 --2- 172.21.15.22:0/779330558 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdfe406a710 0x7fdfd00050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:43.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40cd560 con 0x7fdfd0003a30 2023-07-06T01:19:43.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd000ba40 con 0x7fdfd00043d0 2023-07-06T01:19:43.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd000bc00 con 0x7fdfe406a710 2023-07-06T01:19:43.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfe978b700 1 --2- 172.21.15.22:0/779330558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd0003a30 0x7fdfd0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:43.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfe8f8a700 1 --2- 172.21.15.22:0/779330558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfd00043d0 0x7fdfd0004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:43.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfe9f8c700 1 --2- 172.21.15.22:0/779330558 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdfe406a710 0x7fdfd00050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:43.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfe978b700 1 -- 172.21.15.22:0/779330558 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdfe406a710 msgr2=0x7fdfd00050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:43.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfe978b700 1 --2- 172.21.15.22:0/779330558 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdfe406a710 0x7fdfd00050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:43.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfe978b700 1 -- 172.21.15.22:0/779330558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfd00043d0 msgr2=0x7fdfd0004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:43.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfe978b700 1 --2- 172.21.15.22:0/779330558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfd00043d0 0x7fdfd0004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:43.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfe978b700 1 -- 172.21.15.22:0/779330558 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd00035b0 con 0x7fdfd0003a30 2023-07-06T01:19:43.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.796+0000 7fdfe978b700 1 --2- 172.21.15.22:0/779330558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd0003a30 0x7fdfd0003e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400c240 tx=0x7fdfd0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7ce4d0c334ef9e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:43.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.797+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd400cd80 con 0x7fdfd0003a30 2023-07-06T01:19:43.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.797+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd4005960 con 0x7fdfd0003a30 2023-07-06T01:19:43.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.797+0000 7fdfdbfff700 1 -- 172.21.15.22:0/779330558 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4005d10 con 0x7fdfd0003a30 2023-07-06T01:19:43.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.797+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/779330558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd0003a30 msgr2=0x7fdfd0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:43.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.797+0000 7fdfeb9ef700 1 --2- 172.21.15.22:0/779330558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd0003a30 0x7fdfd0003e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400c240 tx=0x7fdfd0005880 comp rx=0 tx=0).stop 2023-07-06T01:19:43.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.797+0000 7fdfe9f8c700 1 -- 172.21.15.22:0/779330558 reap_dead start 2023-07-06T01:19:43.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.797+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/779330558 shutdown_connections 2023-07-06T01:19:43.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.797+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/779330558 >> 172.21.15.22:0/779330558 conn(0x7fdfe40b1890 msgr2=0x7fdfe40b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:43.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.797+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/779330558 shutdown_connections 2023-07-06T01:19:43.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.797+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/779330558 wait complete. 2023-07-06T01:19:43.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.799+0000 7fdfeb9ef700 1 Processor -- start 2023-07-06T01:19:43.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.799+0000 7fdfeb9ef700 1 -- start start 2023-07-06T01:19:43.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.799+0000 7fdfeb9ef700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdfd0003a30 0x7fdfe415b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:43.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.799+0000 7fdfeb9ef700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe415a0d0 0x7fdfe415a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:43.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.799+0000 7fdfeb9ef700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe40b2b90 0x7fdfe40b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:43.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.799+0000 7fdfeb9ef700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe406e210 con 0x7fdfe415a0d0 2023-07-06T01:19:43.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.799+0000 7fdfeb9ef700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40c8260 con 0x7fdfe40b2b90 2023-07-06T01:19:43.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.799+0000 7fdfeb9ef700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40c8540 con 0x7fdfd0003a30 2023-07-06T01:19:43.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.799+0000 7fdfe8f8a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe415a0d0 0x7fdfe415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:43.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfe8f8a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe415a0d0 0x7fdfe415a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46098/0 (socket says 172.21.15.22:46098) 2023-07-06T01:19:43.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfe8f8a700 1 -- 172.21.15.22:0/1231747520 learned_addr learned my addr 172.21.15.22:0/1231747520 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:43.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfe978b700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdfd0003a30 0x7fdfe415b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:43.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfe9f8c700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe40b2b90 0x7fdfe40b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:43.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfe8f8a700 1 -- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdfd0003a30 msgr2=0x7fdfe415b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:43.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfe8f8a700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdfd0003a30 0x7fdfe415b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:43.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfe8f8a700 1 -- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe40b2b90 msgr2=0x7fdfe40b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:43.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfe8f8a700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe40b2b90 0x7fdfe40b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:43.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfe8f8a700 1 -- 172.21.15.22:0/1231747520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe40c8820 con 0x7fdfe415a0d0 2023-07-06T01:19:43.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfe8f8a700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe415a0d0 0x7fdfe415a540 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdfe00119a0 tx=0x7fdfe0011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:43.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfd9ffb700 1 -- 172.21.15.22:0/1231747520 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000f940 con 0x7fdfe415a0d0 2023-07-06T01:19:43.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfd9ffb700 1 -- 172.21.15.22:0/1231747520 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe000faa0 con 0x7fdfe415a0d0 2023-07-06T01:19:43.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/1231747520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe40c8b00 con 0x7fdfe415a0d0 2023-07-06T01:19:43.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfd9ffb700 1 -- 172.21.15.22:0/1231747520 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000fe30 con 0x7fdfe415a0d0 2023-07-06T01:19:43.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.800+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/1231747520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe415bba0 con 0x7fdfe415a0d0 2023-07-06T01:19:43.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.801+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/1231747520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe40744b0 con 0x7fdfe415a0d0 2023-07-06T01:19:43.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.802+0000 7fdfd9ffb700 1 -- 172.21.15.22:0/1231747520 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdfe001d400 con 0x7fdfe415a0d0 2023-07-06T01:19:43.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.802+0000 7fdfd9ffb700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fdfc806ace0 0x7fdfc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:43.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.802+0000 7fdfd9ffb700 1 -- 172.21.15.22:0/1231747520 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdfe0089b10 con 0x7fdfe415a0d0 2023-07-06T01:19:43.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.802+0000 7fdfe978b700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fdfc806ace0 0x7fdfc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:43.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.803+0000 7fdfe978b700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fdfc806ace0 0x7fdfc806d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4015040 tx=0x7fdfd400a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:43.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:43.807+0000 7fdfd9ffb700 1 -- 172.21.15.22:0/1231747520 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdfe0054bc0 con 0x7fdfe415a0d0 2023-07-06T01:19:44.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.052+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/1231747520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdfe40712e0 con 0x7fdfe415a0d0 2023-07-06T01:19:44.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.052+0000 7fdfd9ffb700 1 -- 172.21.15.22:0/1231747520 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdfe00581e0 con 0x7fdfe415a0d0 2023-07-06T01:19:44.055 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T01:19:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.054+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fdfc806ace0 msgr2=0x7fdfc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.054+0000 7fdfeb9ef700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fdfc806ace0 0x7fdfc806d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4015040 tx=0x7fdfd400a000 comp rx=0 tx=0).stop 2023-07-06T01:19:44.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.054+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/1231747520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe415a0d0 msgr2=0x7fdfe415a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.054+0000 7fdfeb9ef700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe415a0d0 0x7fdfe415a540 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdfe00119a0 tx=0x7fdfe0011d60 comp rx=0 tx=0).stop 2023-07-06T01:19:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.054+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/1231747520 shutdown_connections 2023-07-06T01:19:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.054+0000 7fdfeb9ef700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fdfc806ace0 0x7fdfc806d190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.054+0000 7fdfeb9ef700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfe40b2b90 0x7fdfe40b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.054+0000 7fdfeb9ef700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdfd0003a30 0x7fdfe415b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.054+0000 7fdfeb9ef700 1 --2- 172.21.15.22:0/1231747520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfe415a0d0 0x7fdfe415a540 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.054+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/1231747520 >> 172.21.15.22:0/1231747520 conn(0x7fdfe40b1890 msgr2=0x7fdfe40ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:44.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.054+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/1231747520 shutdown_connections 2023-07-06T01:19:44.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:44.054+0000 7fdfeb9ef700 1 -- 172.21.15.22:0/1231747520 wait complete. 2023-07-06T01:19:44.069 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T01:19:44.069 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T01:19:44.070 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2023-07-06T01:19:44.070 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-07-06T01:19:44.070 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-06T01:19:44.070 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T01:19:44.070 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T01:19:44.071 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T01:19:44.071 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-07-06T01:19:44.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:44.071 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2023-07-06T01:19:44.089 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T01:19:44.089 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T01:19:44.089 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T01:19:44.089 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.089 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-07-06T01:19:44.090 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.090 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:19:44.090 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:44.090 INFO:teuthology.orchestra.run.smithi022.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T01:19:44.090 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:44.090 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-06T01:19:44.090 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42277sec preferred_lft 42277sec 2023-07-06T01:19:44.091 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2023-07-06T01:19:44.091 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.091 INFO:teuthology.orchestra.run.smithi022.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:19:44.091 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:44.091 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:19:44.091 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:44.091 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T01:19:44.091 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 32:fb:2f:9c:8b:1f brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:44.092 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T01:19:44.092 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.092 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::b40c:beff:fe3c:1e8e/64 scope link 2023-07-06T01:19:44.092 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.092 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T01:19:44.092 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 32:fb:2f:9c:8b:1f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T01:19:44.092 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::30fb:2fff:fe9c:8b1f/64 scope link tentative 2023-07-06T01:19:44.092 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:44.093 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T01:19:44.155 INFO:teuthology.orchestra.run.smithi022.stdout:1 2023-07-06T01:19:44.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:44.157 DEBUG:teuthology.orchestra.run.smithi022:> route 2023-07-06T01:19:44.216 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2023-07-06T01:19:44.216 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T01:19:44.216 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-06T01:19:44.216 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-06T01:19:44.217 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T01:19:44.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:44.217 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-07-06T01:19:44.217 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-07-06T01:19:44.218 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-06T01:19:44.218 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-06T01:19:44.218 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-06T01:19:44.218 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-07-06T01:19:44.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:44.323 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-07-06T01:19:44.340 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T01:19:44.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:44.341 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2023-07-06T01:19:44.397 INFO:teuthology.orchestra.run.smithi022.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-07-06T01:19:44.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:44.398 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-07-06T01:19:44.398 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-07-06T01:19:44.398 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T01:19:44.399 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-07-06T01:19:44.399 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-07-06T01:19:44.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:44.493 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-06T01:19:44.532 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T01:19:44.533 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T01:19:44.533 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T01:19:44.534 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.534 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-07-06T01:19:44.534 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.534 INFO:teuthology.orchestra.run.smithi022.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T01:19:44.535 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether ca:2c:65:4e:d2:20 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T01:19:44.535 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T01:19:44.535 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.535 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::c82c:65ff:fe4e:d220/64 scope link tentative 2023-07-06T01:19:44.536 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:44.549 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-06T01:19:44.628 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T01:19:44.629 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T01:19:44.629 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T01:19:44.629 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.629 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-07-06T01:19:44.629 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.629 INFO:teuthology.orchestra.run.smithi022.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T01:19:44.630 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether ca:2c:65:4e:d2:20 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T01:19:44.630 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T01:19:44.630 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.630 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::c82c:65ff:fe4e:d220/64 scope link tentative 2023-07-06T01:19:44.630 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:44.640 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-07-06T01:19:44.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:44.641 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-07-06T01:19:44.642 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-07-06T01:19:44.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-07-06T01:19:44.643 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-06T01:19:44.643 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-07-06T01:19:44.643 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-07-06T01:19:44.644 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-07-06T01:19:44.644 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-07-06T01:19:44.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:44.907 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-07-06T01:19:44.908 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-07-06T01:19:44.908 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.1 up 2023-07-06T01:19:44.909 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.1 master ceph-brx 2023-07-06T01:19:44.909 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-07-06T01:19:45.028 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T01:19:45.029 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-07-06T01:19:45.060 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-07-06T01:19:45.061 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T01:19:45.061 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-07-06T01:19:45.117 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-07-06T01:19:45.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:45.118 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:45.204 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:45.204 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:19:45.233 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:19:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.103:3300,172.21.15.103:3301" 2023-07-06T01:19:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with new device syntax: 2@63a894dd-9157-4ce0-adf3-c1bdfe16ec64.cephfs=/ 2023-07-06T01:19:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.22:3300/172.21.15.103:3300/172.21.15.103:3301" will pass to kernel 2023-07-06T01:19:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: v2 mount support check returned 2 2023-07-06T01:19:45.234 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-06T01:19:45.235 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T01:19:45.308 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:45.308 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:19:45.309 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.103:3300,172.21.15.103:3301" 2023-07-06T01:19:45.309 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with new device syntax: 2@63a894dd-9157-4ce0-adf3-c1bdfe16ec64.cephfs=/ 2023-07-06T01:19:45.309 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.22:3300/172.21.15.103:3300/172.21.15.103:3301" will pass to kernel 2023-07-06T01:19:45.309 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: v2 mount support check returned 2 2023-07-06T01:19:45.309 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-06T01:19:45.309 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T01:19:45.309 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.103:6789,172.21.15.103:6790" 2023-07-06T01:19:45.310 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.103:6789,172.21.15.103:6790:/ 2023-07-06T01:19:45.310 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=63a894dd-9157-4ce0-adf3-c1bdfe16ec64" will pass to kernel 2023-07-06T01:19:45.311 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T01:19:45.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:45.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-07-06T01:19:45.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:45.339 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T01:19:45.411 INFO:teuthology.orchestra.run.smithi022.stdout:client4505 2023-07-06T01:19:45.412 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:45.413 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:19:45.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:45.470 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T01:19:45.542 INFO:teuthology.orchestra.run.smithi022.stdout:client4505 2023-07-06T01:19:45.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:45.542 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T01:19:45.614 INFO:teuthology.orchestra.run.smithi022.stdout:client4505 2023-07-06T01:19:45.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:45.615 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/63a894dd-9157-4ce0-adf3-c1bdfe16ec64.client4505/status') 2023-07-06T01:19:45.696 INFO:teuthology.orchestra.run.smithi022.stdout:instance: client.4505 (3)172.21.15.22:0/2982616989 2023-07-06T01:19:45.696 INFO:teuthology.orchestra.run.smithi022.stdout:blocklisted: false 2023-07-06T01:19:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records in 2023-07-06T01:19:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records out 2023-07-06T01:19:45.698 INFO:teuthology.orchestra.run.smithi022.stderr:70 bytes copied, 2.278e-05 s, 3.1 MB/s 2023-07-06T01:19:45.699 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-07-06T01:19:45.699 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T01:19:45.699 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T01:19:45.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T01:19:45.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.860+0000 7f693ae80700 1 Processor -- start 2023-07-06T01:19:45.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.860+0000 7f693ae80700 1 -- start start 2023-07-06T01:19:45.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.861+0000 7f693ae80700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f693415a780 0x7f693406a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:45.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.861+0000 7f693ae80700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f693415f100 0x7f693406a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:45.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.861+0000 7f693ae80700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f693415e3d0 con 0x7f69340cfbf0 2023-07-06T01:19:45.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.861+0000 7f693ae80700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69340cc4b0 con 0x7f693415f100 2023-07-06T01:19:45.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.861+0000 7f693ae80700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f693415ec70 con 0x7f693415a780 2023-07-06T01:19:45.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.861+0000 7f6933fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f693415f100 0x7f693406a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:45.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.861+0000 7f693941d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f69340cfbf0 0x7f69340d0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36852/0 (socket says 172.21.15.22:36852) 2023-07-06T01:19:45.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.861+0000 7f693941d700 1 -- 172.21.15.22:0/3153307063 learned_addr learned my addr 172.21.15.22:0/3153307063 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:45.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.862+0000 7f6938c1c700 1 --2- 172.21.15.22:0/3153307063 >> v2:172.21.15.103:3300/0 conn(0x7f693415a780 0x7f693406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:45.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.862+0000 7f6933fff700 1 -- 172.21.15.22:0/3153307063 >> v2:172.21.15.103:3300/0 conn(0x7f693415a780 msgr2=0x7f693406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:45.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.862+0000 7f6933fff700 1 --2- 172.21.15.22:0/3153307063 >> v2:172.21.15.103:3300/0 conn(0x7f693415a780 0x7f693406a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:45.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.862+0000 7f6933fff700 1 -- 172.21.15.22:0/3153307063 >> v1:172.21.15.22:6789/0 conn(0x7f69340cfbf0 legacy=0x7f69340d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:45.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.862+0000 7f6933fff700 1 -- 172.21.15.22:0/3153307063 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69340d0920 con 0x7f693415f100 2023-07-06T01:19:45.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.862+0000 7f69337fe700 1 -- 172.21.15.22:0/3153307063 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574099194 0 0) 0x7f693415e3d0 con 0x7f69340cfbf0 2023-07-06T01:19:45.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.862+0000 7f6938c1c700 1 --2- 172.21.15.22:0/3153307063 >> v2:172.21.15.103:3300/0 conn(0x7f693415a780 0x7f693406a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:45.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.862+0000 7f6933fff700 1 --2- 172.21.15.22:0/3153307063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f693415f100 0x7f693406a540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69340d0d00 tx=0x7f6920009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121dcc8d7c49dda3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:45.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.862+0000 7f69337fe700 1 -- 172.21.15.22:0/3153307063 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920011830 con 0x7f693415f100 2023-07-06T01:19:45.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.862+0000 7f69337fe700 1 -- 172.21.15.22:0/3153307063 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6920011990 con 0x7f693415f100 2023-07-06T01:19:45.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.862+0000 7f69337fe700 1 -- 172.21.15.22:0/3153307063 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920011d20 con 0x7f693415f100 2023-07-06T01:19:45.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.863+0000 7f693ae80700 1 -- 172.21.15.22:0/3153307063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f693415f100 msgr2=0x7f693406a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:45.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.863+0000 7f693ae80700 1 --2- 172.21.15.22:0/3153307063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f693415f100 0x7f693406a540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69340d0d00 tx=0x7f6920009b20 comp rx=0 tx=0).stop 2023-07-06T01:19:45.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.863+0000 7f693ae80700 1 -- 172.21.15.22:0/3153307063 shutdown_connections 2023-07-06T01:19:45.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.863+0000 7f693ae80700 1 --2- 172.21.15.22:0/3153307063 >> v2:172.21.15.103:3300/0 conn(0x7f693415a780 0x7f693406a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:45.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.863+0000 7f693ae80700 1 --2- 172.21.15.22:0/3153307063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f693415f100 0x7f693406a540 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.863+0000 7f693ae80700 1 -- 172.21.15.22:0/3153307063 >> 172.21.15.22:0/3153307063 conn(0x7f693406b110 msgr2=0x7f693406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.863+0000 7f693ae80700 1 -- 172.21.15.22:0/3153307063 shutdown_connections 2023-07-06T01:19:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.863+0000 7f693ae80700 1 -- 172.21.15.22:0/3153307063 wait complete. 2023-07-06T01:19:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.864+0000 7f693ae80700 1 Processor -- start 2023-07-06T01:19:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.864+0000 7f693ae80700 1 -- start start 2023-07-06T01:19:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f693ae80700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69340cfbf0 0x7f6934151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f693ae80700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f693415a780 0x7f693406cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f693ae80700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f693415f100 0x7f693406d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f693ae80700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693415d1f0 con 0x7f693415a780 2023-07-06T01:19:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f693ae80700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934151fa0 con 0x7f693415f100 2023-07-06T01:19:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f693ae80700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693415c5e0 con 0x7f69340cfbf0 2023-07-06T01:19:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f6933fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f693415a780 0x7f693406cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f6933fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f693415a780 0x7f693406cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46116/0 (socket says 172.21.15.22:46116) 2023-07-06T01:19:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f6933fff700 1 -- 172.21.15.22:0/2161078073 learned_addr learned my addr 172.21.15.22:0/2161078073 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f693941d700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f693415f100 0x7f693406d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f6938c1c700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69340cfbf0 0x7f6934151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f6933fff700 1 -- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69340cfbf0 msgr2=0x7f6934151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f6933fff700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69340cfbf0 0x7f6934151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f6933fff700 1 -- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f693415f100 msgr2=0x7f693406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f6933fff700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f693415f100 0x7f693406d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f6933fff700 1 -- 172.21.15.22:0/2161078073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200094a0 con 0x7f693415a780 2023-07-06T01:19:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f6933fff700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f693415a780 0x7f693406cae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6920000c00 tx=0x7f6920004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:45.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.865+0000 7f69317fa700 1 -- 172.21.15.22:0/2161078073 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69200036a0 con 0x7f693415a780 2023-07-06T01:19:45.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.866+0000 7f69317fa700 1 -- 172.21.15.22:0/2161078073 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692001e070 con 0x7f693415a780 2023-07-06T01:19:45.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.866+0000 7f693ae80700 1 -- 172.21.15.22:0/2161078073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693415c8c0 con 0x7f693415a780 2023-07-06T01:19:45.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.866+0000 7f69317fa700 1 -- 172.21.15.22:0/2161078073 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920003a20 con 0x7f693415a780 2023-07-06T01:19:45.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.866+0000 7f693ae80700 1 -- 172.21.15.22:0/2161078073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693415ce80 con 0x7f693415a780 2023-07-06T01:19:45.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.867+0000 7f693ae80700 1 -- 172.21.15.22:0/2161078073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f693415d360 con 0x7f693415a780 2023-07-06T01:19:45.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.867+0000 7f69317fa700 1 -- 172.21.15.22:0/2161078073 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6920003b80 con 0x7f693415a780 2023-07-06T01:19:45.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.868+0000 7f69317fa700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f691806ad10 0x7f691806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:45.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.868+0000 7f69317fa700 1 -- 172.21.15.22:0/2161078073 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f692008a4d0 con 0x7f693415a780 2023-07-06T01:19:45.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.868+0000 7f6938c1c700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f691806ad10 0x7f691806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:45.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.868+0000 7f6938c1c700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f691806ad10 0x7f691806d1c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6928006fd0 tx=0x7f6928008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:45.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:45.872+0000 7f69317fa700 1 -- 172.21.15.22:0/2161078073 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6920055570 con 0x7f693415a780 2023-07-06T01:19:46.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.118+0000 7f693ae80700 1 -- 172.21.15.22:0/2161078073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f69340744b0 con 0x7f693415a780 2023-07-06T01:19:46.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.118+0000 7f69317fa700 1 -- 172.21.15.22:0/2161078073 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6920058b90 con 0x7f693415a780 2023-07-06T01:19:46.121 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T01:19:46.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.120+0000 7f693ae80700 1 -- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f691806ad10 msgr2=0x7f691806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:46.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.120+0000 7f693ae80700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f691806ad10 0x7f691806d1c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6928006fd0 tx=0x7f6928008040 comp rx=0 tx=0).stop 2023-07-06T01:19:46.123 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.120+0000 7f693ae80700 1 -- 172.21.15.22:0/2161078073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f693415a780 msgr2=0x7f693406cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:46.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.120+0000 7f693ae80700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f693415a780 0x7f693406cae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6920000c00 tx=0x7f6920004750 comp rx=0 tx=0).stop 2023-07-06T01:19:46.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.120+0000 7f693ae80700 1 -- 172.21.15.22:0/2161078073 shutdown_connections 2023-07-06T01:19:46.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.120+0000 7f693ae80700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f691806ad10 0x7f691806d1c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:46.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.120+0000 7f693ae80700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f693415f100 0x7f693406d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:46.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.120+0000 7f693ae80700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69340cfbf0 0x7f6934151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:46.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.120+0000 7f693ae80700 1 --2- 172.21.15.22:0/2161078073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f693415a780 0x7f693406cae0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:46.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.120+0000 7f693ae80700 1 -- 172.21.15.22:0/2161078073 >> 172.21.15.22:0/2161078073 conn(0x7f693406b110 msgr2=0x7f69340b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:46.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.121+0000 7f693ae80700 1 -- 172.21.15.22:0/2161078073 shutdown_connections 2023-07-06T01:19:46.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:46.121+0000 7f693ae80700 1 -- 172.21.15.22:0/2161078073 wait complete. 2023-07-06T01:19:46.135 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T01:19:46.136 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T01:19:46.136 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi103.front.sepia.ceph.com 2023-07-06T01:19:46.136 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T01:19:46.136 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T01:19:46.136 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T01:19:46.136 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T01:19:46.137 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T01:19:46.137 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T01:19:46.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:46.137 DEBUG:teuthology.orchestra.run.smithi103:> ip addr 2023-07-06T01:19:46.158 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T01:19:46.158 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T01:19:46.158 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T01:19:46.159 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:46.159 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2023-07-06T01:19:46.159 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:46.159 INFO:teuthology.orchestra.run.smithi103.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:19:46.159 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c4 brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:46.159 INFO:teuthology.orchestra.run.smithi103.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:19:46.159 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c5 brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:46.159 INFO:teuthology.orchestra.run.smithi103.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:19:46.160 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0a brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:46.160 INFO:teuthology.orchestra.run.smithi103.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T01:19:46.160 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0b brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:46.160 INFO:teuthology.orchestra.run.smithi103.stdout: inet 172.21.15.103/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T01:19:46.160 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft 42312sec preferred_lft 42312sec 2023-07-06T01:19:46.160 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::b0c1:78c3:def8:9098/64 scope link noprefixroute 2023-07-06T01:19:46.160 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:46.161 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-06T01:19:46.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:46.161 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec bash -c ' 2023-07-06T01:19:46.162 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2023-07-06T01:19:46.162 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add name ceph-brx type bridge 2023-07-06T01:19:46.162 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr flush dev ceph-brx 2023-07-06T01:19:46.162 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set ceph-brx up 2023-07-06T01:19:46.162 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-06T01:19:46.163 DEBUG:teuthology.orchestra.run.smithi103:> ') 2023-07-06T01:19:46.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:46.338 DEBUG:teuthology.orchestra.run.smithi103:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T01:19:46.365 INFO:teuthology.orchestra.run.smithi103.stdout:1 2023-07-06T01:19:46.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:46.367 DEBUG:teuthology.orchestra.run.smithi103:> route 2023-07-06T01:19:46.435 INFO:teuthology.orchestra.run.smithi103.stdout:Kernel IP routing table 2023-07-06T01:19:46.435 INFO:teuthology.orchestra.run.smithi103.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T01:19:46.436 INFO:teuthology.orchestra.run.smithi103.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T01:19:46.436 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T01:19:46.436 INFO:teuthology.orchestra.run.smithi103.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T01:19:46.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:46.437 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec bash -c ' 2023-07-06T01:19:46.437 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2023-07-06T01:19:46.437 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T01:19:46.437 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T01:19:46.437 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T01:19:46.438 DEBUG:teuthology.orchestra.run.smithi103:> ') 2023-07-06T01:19:46.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:46.699 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2023-07-06T01:19:46.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:46.715 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list-id 2023-07-06T01:19:46.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:46.772 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec bash -c ' 2023-07-06T01:19:46.773 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2023-07-06T01:19:46.773 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T01:19:46.773 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-07-06T01:19:46.773 DEBUG:teuthology.orchestra.run.smithi103:> ') 2023-07-06T01:19:46.865 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-07-06T01:19:46.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:46.866 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec bash -c ' 2023-07-06T01:19:46.866 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2023-07-06T01:19:46.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-07-06T01:19:46.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-06T01:19:46.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-07-06T01:19:46.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-07-06T01:19:46.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-07-06T01:19:46.868 DEBUG:teuthology.orchestra.run.smithi103:> ') 2023-07-06T01:19:47.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:47.074 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec bash -c ' 2023-07-06T01:19:47.074 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2023-07-06T01:19:47.074 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set brx.0 up 2023-07-06T01:19:47.074 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set dev brx.0 master ceph-brx 2023-07-06T01:19:47.075 DEBUG:teuthology.orchestra.run.smithi103:> ') 2023-07-06T01:19:47.174 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T01:19:47.174 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T01:19:47.195 INFO:teuthology.orchestra.run.smithi103.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T01:19:47.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T01:19:47.196 DEBUG:teuthology.orchestra.run.smithi103:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T01:19:47.252 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-07-06T01:19:47.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:47.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:47.343 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:47.344 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:19:47.457 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:47.457 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:19:47.457 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.103:3300,172.21.15.103:3301" 2023-07-06T01:19:47.458 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: trying mount with new device syntax: 1@63a894dd-9157-4ce0-adf3-c1bdfe16ec64.cephfs=/ 2023-07-06T01:19:47.458 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.22:3300/172.21.15.103:3300/172.21.15.103:3301" will pass to kernel 2023-07-06T01:19:47.458 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: v2 mount support check returned 2 2023-07-06T01:19:47.458 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-06T01:19:47.458 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T01:19:47.481 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:47.481 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:19:47.481 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.103:3300,172.21.15.103:3301" 2023-07-06T01:19:47.481 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: trying mount with new device syntax: 1@63a894dd-9157-4ce0-adf3-c1bdfe16ec64.cephfs=/ 2023-07-06T01:19:47.481 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.22:3300/172.21.15.103:3300/172.21.15.103:3301" will pass to kernel 2023-07-06T01:19:47.482 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: v2 mount support check returned 2 2023-07-06T01:19:47.482 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-06T01:19:47.482 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T01:19:47.482 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.103:6789,172.21.15.103:6790" 2023-07-06T01:19:47.482 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.103:6789,172.21.15.103:6790:/ 2023-07-06T01:19:47.482 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=63a894dd-9157-4ce0-adf3-c1bdfe16ec64" will pass to kernel 2023-07-06T01:19:47.484 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T01:19:47.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:47.485 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T01:19:47.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:47.515 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T01:19:47.586 INFO:teuthology.orchestra.run.smithi103.stdout:client4511 2023-07-06T01:19:47.586 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:47.587 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:19:47.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:47.645 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T01:19:47.716 INFO:teuthology.orchestra.run.smithi103.stdout:client4511 2023-07-06T01:19:47.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:47.717 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T01:19:47.788 INFO:teuthology.orchestra.run.smithi103.stdout:client4511 2023-07-06T01:19:47.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:47.789 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/63a894dd-9157-4ce0-adf3-c1bdfe16ec64.client4511/status') 2023-07-06T01:19:47.868 INFO:teuthology.orchestra.run.smithi103.stdout:instance: client.4511 (3)192.168.0.1:0/3807284102 2023-07-06T01:19:47.868 INFO:teuthology.orchestra.run.smithi103.stdout:blocklisted: false 2023-07-06T01:19:47.869 INFO:teuthology.orchestra.run.smithi103.stderr:0+1 records in 2023-07-06T01:19:47.869 INFO:teuthology.orchestra.run.smithi103.stderr:0+1 records out 2023-07-06T01:19:47.869 INFO:teuthology.orchestra.run.smithi103.stderr:69 bytes copied, 2.2176e-05 s, 3.1 MB/s 2023-07-06T01:19:47.870 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-07-06T01:19:47.870 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T01:19:47.870 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-06T01:19:47.871 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T01:19:48.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.007+0000 7f3ea7731700 1 Processor -- start 2023-07-06T01:19:48.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.007+0000 7f3ea7731700 1 -- start start 2023-07-06T01:19:48.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3ea7731700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3ea00af2a0 0x7f3ea006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:48.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3ea7731700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3ea006a000 0x7f3ea006a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:48.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3ea7731700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea00af670 con 0x7f3ea00cbc10 2023-07-06T01:19:48.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3ea7731700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea006a9e0 con 0x7f3ea00af2a0 2023-07-06T01:19:48.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3ea7731700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea00cd560 con 0x7f3ea006a000 2023-07-06T01:19:48.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3ea54cd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f3ea00af2a0 0x7f3ea006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:48.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3ea5cce700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f3ea00cbc10 0x7f3ea00cc050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36870/0 (socket says 172.21.15.22:36870) 2023-07-06T01:19:48.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3ea5cce700 1 -- 172.21.15.22:0/2426981536 learned_addr learned my addr 172.21.15.22:0/2426981536 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:48.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3ea4ccc700 1 --2- 172.21.15.22:0/2426981536 >> v2:172.21.15.103:3300/0 conn(0x7f3ea006a000 0x7f3ea006a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:48.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3e97fff700 1 -- 172.21.15.22:0/2426981536 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 97387263 0 0) 0x7f3ea00af670 con 0x7f3ea00cbc10 2023-07-06T01:19:48.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3e97fff700 1 -- 172.21.15.22:0/2426981536 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e8c0035b0 con 0x7f3ea00cbc10 2023-07-06T01:19:48.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3ea54cd700 1 -- 172.21.15.22:0/2426981536 >> v2:172.21.15.103:3300/0 conn(0x7f3ea006a000 msgr2=0x7f3ea006a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:48.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3ea54cd700 1 --2- 172.21.15.22:0/2426981536 >> v2:172.21.15.103:3300/0 conn(0x7f3ea006a000 0x7f3ea006a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:48.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3ea54cd700 1 -- 172.21.15.22:0/2426981536 >> v1:172.21.15.22:6789/0 conn(0x7f3ea00cbc10 legacy=0x7f3ea00cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:48.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.008+0000 7f3ea54cd700 1 -- 172.21.15.22:0/2426981536 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea006d030 con 0x7f3ea00af2a0 2023-07-06T01:19:48.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.009+0000 7f3ea54cd700 1 --2- 172.21.15.22:0/2426981536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea00af2a0 0x7f3ea006caf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0071b20 tx=0x7f3e9c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c97babd488d1b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:48.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.009+0000 7f3e97fff700 1 -- 172.21.15.22:0/2426981536 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c011830 con 0x7f3ea00af2a0 2023-07-06T01:19:48.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.009+0000 7f3e97fff700 1 -- 172.21.15.22:0/2426981536 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c011990 con 0x7f3ea00af2a0 2023-07-06T01:19:48.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.009+0000 7f3e97fff700 1 -- 172.21.15.22:0/2426981536 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c011d20 con 0x7f3ea00af2a0 2023-07-06T01:19:48.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.009+0000 7f3ea7731700 1 -- 172.21.15.22:0/2426981536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea00af2a0 msgr2=0x7f3ea006caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:48.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.009+0000 7f3ea7731700 1 --2- 172.21.15.22:0/2426981536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea00af2a0 0x7f3ea006caf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0071b20 tx=0x7f3e9c009b20 comp rx=0 tx=0).stop 2023-07-06T01:19:48.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.009+0000 7f3ea7731700 1 -- 172.21.15.22:0/2426981536 shutdown_connections 2023-07-06T01:19:48.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.009+0000 7f3ea7731700 1 --2- 172.21.15.22:0/2426981536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea00af2a0 0x7f3ea006caf0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:48.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.009+0000 7f3ea7731700 1 --2- 172.21.15.22:0/2426981536 >> v2:172.21.15.103:3300/0 conn(0x7f3ea006a000 0x7f3ea006a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:48.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.009+0000 7f3ea7731700 1 -- 172.21.15.22:0/2426981536 >> 172.21.15.22:0/2426981536 conn(0x7f3ea00b1890 msgr2=0x7f3ea00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:48.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.009+0000 7f3ea7731700 1 -- 172.21.15.22:0/2426981536 shutdown_connections 2023-07-06T01:19:48.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.009+0000 7f3ea7731700 1 -- 172.21.15.22:0/2426981536 wait complete. 2023-07-06T01:19:48.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.011+0000 7f3ea7731700 1 Processor -- start 2023-07-06T01:19:48.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.011+0000 7f3ea7731700 1 -- start start 2023-07-06T01:19:48.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.011+0000 7f3ea7731700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea006a000 0x7f3ea01f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:48.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.011+0000 7f3ea7731700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea00af2a0 0x7f3ea015ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:48.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.011+0000 7f3ea7731700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3ea00cbc10 0x7f3ea01477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:48.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.011+0000 7f3ea7731700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea015f490 con 0x7f3ea00af2a0 2023-07-06T01:19:48.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.011+0000 7f3ea7731700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01e5650 con 0x7f3ea006a000 2023-07-06T01:19:48.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.011+0000 7f3ea7731700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea015a900 con 0x7f3ea00cbc10 2023-07-06T01:19:48.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea4ccc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea00af2a0 0x7f3ea015ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:48.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea4ccc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea00af2a0 0x7f3ea015ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46130/0 (socket says 172.21.15.22:46130) 2023-07-06T01:19:48.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea4ccc700 1 -- 172.21.15.22:0/3061955333 learned_addr learned my addr 172.21.15.22:0/3061955333 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:48.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea54cd700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea006a000 0x7f3ea01f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:48.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea5cce700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3ea00cbc10 0x7f3ea01477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:48.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea4ccc700 1 -- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3ea00cbc10 msgr2=0x7f3ea01477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:48.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea4ccc700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3ea00cbc10 0x7f3ea01477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea4ccc700 1 -- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea006a000 msgr2=0x7f3ea01f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea4ccc700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea006a000 0x7f3ea01f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea4ccc700 1 -- 172.21.15.22:0/3061955333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c0094a0 con 0x7f3ea00af2a0 2023-07-06T01:19:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea54cd700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea006a000 0x7f3ea01f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea5cce700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3ea00cbc10 0x7f3ea01477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea4ccc700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea00af2a0 0x7f3ea015ccc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e9e0 tx=0x7f3e9000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3e95ffb700 1 -- 172.21.15.22:0/3061955333 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90009710 con 0x7f3ea00af2a0 2023-07-06T01:19:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3e95ffb700 1 -- 172.21.15.22:0/3061955333 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9000c850 con 0x7f3ea00af2a0 2023-07-06T01:19:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3ea7731700 1 -- 172.21.15.22:0/3061955333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea01e5320 con 0x7f3ea00af2a0 2023-07-06T01:19:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.012+0000 7f3e95ffb700 1 -- 172.21.15.22:0/3061955333 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000cbc0 con 0x7f3ea00af2a0 2023-07-06T01:19:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.013+0000 7f3ea7731700 1 -- 172.21.15.22:0/3061955333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea015f6e0 con 0x7f3ea00af2a0 2023-07-06T01:19:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.013+0000 7f3ea7731700 1 -- 172.21.15.22:0/3061955333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea00744b0 con 0x7f3ea00af2a0 2023-07-06T01:19:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.014+0000 7f3e95ffb700 1 -- 172.21.15.22:0/3061955333 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e9000cd20 con 0x7f3ea00af2a0 2023-07-06T01:19:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.014+0000 7f3e95ffb700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3e8406ace0 0x7f3e8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.014+0000 7f3e95ffb700 1 -- 172.21.15.22:0/3061955333 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e900899e0 con 0x7f3ea00af2a0 2023-07-06T01:19:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.014+0000 7f3ea54cd700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3e8406ace0 0x7f3e8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.015+0000 7f3ea54cd700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3e8406ace0 0x7f3e8406d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0090b0 tx=0x7f3e9c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:48.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.019+0000 7f3e95ffb700 1 -- 172.21.15.22:0/3061955333 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3e90058480 con 0x7f3ea00af2a0 2023-07-06T01:19:48.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.263+0000 7f3ea7731700 1 -- 172.21.15.22:0/3061955333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3ea015ca90 con 0x7f3ea00af2a0 2023-07-06T01:19:48.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.264+0000 7f3e95ffb700 1 -- 172.21.15.22:0/3061955333 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3ea015ca90 con 0x7f3ea00af2a0 2023-07-06T01:19:48.267 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T01:19:48.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.266+0000 7f3ea7731700 1 -- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3e8406ace0 msgr2=0x7f3e8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:48.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.266+0000 7f3ea7731700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3e8406ace0 0x7f3e8406d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0090b0 tx=0x7f3e9c008030 comp rx=0 tx=0).stop 2023-07-06T01:19:48.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.266+0000 7f3ea7731700 1 -- 172.21.15.22:0/3061955333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea00af2a0 msgr2=0x7f3ea015ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:48.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.266+0000 7f3ea7731700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea00af2a0 0x7f3ea015ccc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000e9e0 tx=0x7f3e9000eda0 comp rx=0 tx=0).stop 2023-07-06T01:19:48.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.266+0000 7f3ea7731700 1 -- 172.21.15.22:0/3061955333 shutdown_connections 2023-07-06T01:19:48.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.266+0000 7f3ea7731700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3ea00cbc10 0x7f3ea01477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:48.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.266+0000 7f3ea7731700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3e8406ace0 0x7f3e8406d190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:48.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.266+0000 7f3ea7731700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ea006a000 0x7f3ea01f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:48.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.266+0000 7f3ea7731700 1 --2- 172.21.15.22:0/3061955333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ea00af2a0 0x7f3ea015ccc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:48.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.266+0000 7f3ea7731700 1 -- 172.21.15.22:0/3061955333 >> 172.21.15.22:0/3061955333 conn(0x7f3ea00b1890 msgr2=0x7f3ea00ada90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:48.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.266+0000 7f3ea7731700 1 -- 172.21.15.22:0/3061955333 shutdown_connections 2023-07-06T01:19:48.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:48.266+0000 7f3ea7731700 1 -- 172.21.15.22:0/3061955333 wait complete. 2023-07-06T01:19:48.281 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T01:19:48.281 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T01:19:48.281 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi103.front.sepia.ceph.com 2023-07-06T01:19:48.282 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-07-06T01:19:48.282 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-06T01:19:48.282 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T01:19:48.282 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T01:19:48.282 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T01:19:48.282 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-07-06T01:19:48.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:48.283 DEBUG:teuthology.orchestra.run.smithi103:> ip addr 2023-07-06T01:19:48.301 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T01:19:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T01:19:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T01:19:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2023-07-06T01:19:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.302 INFO:teuthology.orchestra.run.smithi103.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:19:48.302 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c4 brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:48.303 INFO:teuthology.orchestra.run.smithi103.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:19:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c5 brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:48.303 INFO:teuthology.orchestra.run.smithi103.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:19:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0a brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:48.303 INFO:teuthology.orchestra.run.smithi103.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T01:19:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0b brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:48.303 INFO:teuthology.orchestra.run.smithi103.stdout: inet 172.21.15.103/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T01:19:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft 42310sec preferred_lft 42310sec 2023-07-06T01:19:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::b0c1:78c3:def8:9098/64 scope link noprefixroute 2023-07-06T01:19:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.304 INFO:teuthology.orchestra.run.smithi103.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T01:19:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0a:0f:f7:27:02:f5 brd ff:ff:ff:ff:ff:ff 2023-07-06T01:19:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T01:19:48.304 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::d0e8:22ff:fe5c:241f/64 scope link 2023-07-06T01:19:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.305 INFO:teuthology.orchestra.run.smithi103.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T01:19:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0a:0f:f7:27:02:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T01:19:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::80f:f7ff:fe27:2f5/64 scope link tentative 2023-07-06T01:19:48.305 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:48.306 DEBUG:teuthology.orchestra.run.smithi103:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T01:19:48.367 INFO:teuthology.orchestra.run.smithi103.stdout:1 2023-07-06T01:19:48.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:48.369 DEBUG:teuthology.orchestra.run.smithi103:> route 2023-07-06T01:19:48.427 INFO:teuthology.orchestra.run.smithi103.stdout:Kernel IP routing table 2023-07-06T01:19:48.427 INFO:teuthology.orchestra.run.smithi103.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T01:19:48.427 INFO:teuthology.orchestra.run.smithi103.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T01:19:48.427 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T01:19:48.428 INFO:teuthology.orchestra.run.smithi103.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T01:19:48.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:48.428 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec bash -c ' 2023-07-06T01:19:48.429 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2023-07-06T01:19:48.429 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T01:19:48.429 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T01:19:48.429 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T01:19:48.430 DEBUG:teuthology.orchestra.run.smithi103:> ') 2023-07-06T01:19:48.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:48.532 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2023-07-06T01:19:48.549 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T01:19:48.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:48.551 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list-id 2023-07-06T01:19:48.606 INFO:teuthology.orchestra.run.smithi103.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-07-06T01:19:48.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:48.607 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec bash -c ' 2023-07-06T01:19:48.607 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2023-07-06T01:19:48.607 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T01:19:48.607 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-07-06T01:19:48.607 DEBUG:teuthology.orchestra.run.smithi103:> ') 2023-07-06T01:19:48.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:48.699 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-06T01:19:48.733 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T01:19:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T01:19:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T01:19:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2023-07-06T01:19:48.734 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.735 INFO:teuthology.orchestra.run.smithi103.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T01:19:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 26:26:1b:44:49:dc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T01:19:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T01:19:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::2426:1bff:fe44:49dc/64 scope link 2023-07-06T01:19:48.735 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:48.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-06T01:19:48.830 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T01:19:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T01:19:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T01:19:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2023-07-06T01:19:48.831 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.832 INFO:teuthology.orchestra.run.smithi103.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T01:19:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 26:26:1b:44:49:dc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T01:19:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T01:19:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::2426:1bff:fe44:49dc/64 scope link 2023-07-06T01:19:48.832 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:19:48.843 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-07-06T01:19:48.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:48.844 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec bash -c ' 2023-07-06T01:19:48.844 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2023-07-06T01:19:48.845 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-07-06T01:19:48.845 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-06T01:19:48.845 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-07-06T01:19:48.845 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-07-06T01:19:48.846 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-07-06T01:19:48.846 DEBUG:teuthology.orchestra.run.smithi103:> ') 2023-07-06T01:19:49.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:49.095 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec bash -c ' 2023-07-06T01:19:49.095 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2023-07-06T01:19:49.095 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set brx.1 up 2023-07-06T01:19:49.096 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set dev brx.1 master ceph-brx 2023-07-06T01:19:49.096 DEBUG:teuthology.orchestra.run.smithi103:> ') 2023-07-06T01:19:49.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T01:19:49.205 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-07-06T01:19:49.224 INFO:teuthology.orchestra.run.smithi103.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-07-06T01:19:49.225 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T01:19:49.225 DEBUG:teuthology.orchestra.run.smithi103:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-07-06T01:19:49.282 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-07-06T01:19:49.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:49.283 DEBUG:teuthology.orchestra.run.smithi103:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:49.367 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:49.368 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:19:49.391 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:49.391 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:19:49.391 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.103:3300,172.21.15.103:3301" 2023-07-06T01:19:49.391 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: trying mount with new device syntax: 3@63a894dd-9157-4ce0-adf3-c1bdfe16ec64.cephfs=/ 2023-07-06T01:19:49.391 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.22:3300/172.21.15.103:3300/172.21.15.103:3301" will pass to kernel 2023-07-06T01:19:49.392 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: v2 mount support check returned 2 2023-07-06T01:19:49.392 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-06T01:19:49.392 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T01:19:49.466 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:19:49.466 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:19:49.466 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.103:3300,172.21.15.103:3301" 2023-07-06T01:19:49.466 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: trying mount with new device syntax: 3@63a894dd-9157-4ce0-adf3-c1bdfe16ec64.cephfs=/ 2023-07-06T01:19:49.467 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.22:3300/172.21.15.103:3300/172.21.15.103:3301" will pass to kernel 2023-07-06T01:19:49.467 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: v2 mount support check returned 2 2023-07-06T01:19:49.467 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-06T01:19:49.467 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T01:19:49.467 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.103:6789,172.21.15.103:6790" 2023-07-06T01:19:49.467 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.103:6789,172.21.15.103:6790:/ 2023-07-06T01:19:49.467 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=63a894dd-9157-4ce0-adf3-c1bdfe16ec64" will pass to kernel 2023-07-06T01:19:49.469 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T01:19:49.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:49.470 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-07-06T01:19:49.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:49.496 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T01:19:49.568 INFO:teuthology.orchestra.run.smithi103.stdout:client4674 2023-07-06T01:19:49.568 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:19:49.569 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:19:49.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:49.625 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T01:19:49.698 INFO:teuthology.orchestra.run.smithi103.stdout:client4674 2023-07-06T01:19:49.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:49.699 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T01:19:49.771 INFO:teuthology.orchestra.run.smithi103.stdout:client4674 2023-07-06T01:19:49.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:19:49.772 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/63a894dd-9157-4ce0-adf3-c1bdfe16ec64.client4674/status') 2023-07-06T01:19:49.853 INFO:teuthology.orchestra.run.smithi103.stdout:instance: client.4674 (3)172.21.15.103:0/4095351745 2023-07-06T01:19:49.854 INFO:teuthology.orchestra.run.smithi103.stdout:blocklisted: false 2023-07-06T01:19:49.854 INFO:teuthology.orchestra.run.smithi103.stderr:0+1 records in 2023-07-06T01:19:49.855 INFO:teuthology.orchestra.run.smithi103.stderr:0+1 records out 2023-07-06T01:19:49.855 INFO:teuthology.orchestra.run.smithi103.stderr:71 bytes copied, 2.2137e-05 s, 3.2 MB/s 2023-07-06T01:19:49.855 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-07-06T01:19:49.872 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:50.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.009+0000 7f05d9514700 1 Processor -- start 2023-07-06T01:19:50.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.009+0000 7f05d9514700 1 -- start start 2023-07-06T01:19:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.010+0000 7f05d9514700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05d4159870 0x7f05d415df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.010+0000 7f05d9514700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f05d4071a60 0x7f05d40ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.010+0000 7f05d9514700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05d40ce550 con 0x7f05d40715f0 2023-07-06T01:19:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.010+0000 7f05d9514700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d40cdb60 con 0x7f05d4071a60 2023-07-06T01:19:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.010+0000 7f05d9514700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d415d490 con 0x7f05d4159870 2023-07-06T01:19:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.010+0000 7f05d27fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f05d40715f0 0x7f05d40cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36890/0 (socket says 172.21.15.22:36890) 2023-07-06T01:19:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.010+0000 7f05d27fc700 1 -- 172.21.15.22:0/1309247531 learned_addr learned my addr 172.21.15.22:0/1309247531 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.010+0000 7f05d37fe700 1 --2- 172.21.15.22:0/1309247531 >> v2:172.21.15.103:3300/0 conn(0x7f05d4071a60 0x7f05d40ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.010+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/1309247531 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05d4159870 0x7f05d415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.010+0000 7f05d1ffb700 1 -- 172.21.15.22:0/1309247531 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199400656 0 0) 0x7f05d40ce550 con 0x7f05d40715f0 2023-07-06T01:19:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.010+0000 7f05d1ffb700 1 -- 172.21.15.22:0/1309247531 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05bc0035b0 con 0x7f05d40715f0 2023-07-06T01:19:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.011+0000 7f05d2ffd700 1 -- 172.21.15.22:0/1309247531 >> v2:172.21.15.103:3300/0 conn(0x7f05d4071a60 msgr2=0x7f05d40ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.011+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/1309247531 >> v2:172.21.15.103:3300/0 conn(0x7f05d4071a60 0x7f05d40ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.011+0000 7f05d2ffd700 1 -- 172.21.15.22:0/1309247531 >> v1:172.21.15.22:6789/0 conn(0x7f05d40715f0 legacy=0x7f05d40cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.011+0000 7f05d2ffd700 1 -- 172.21.15.22:0/1309247531 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d415d770 con 0x7f05d4159870 2023-07-06T01:19:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.011+0000 7f05d37fe700 1 --2- 172.21.15.22:0/1309247531 >> v2:172.21.15.103:3300/0 conn(0x7f05d4071a60 0x7f05d40ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.011+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/1309247531 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05d4159870 0x7f05d415df70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009840 tx=0x7f05c8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6aed723e6a4f8b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.011+0000 7f05d1ffb700 1 -- 172.21.15.22:0/1309247531 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8011830 con 0x7f05d4159870 2023-07-06T01:19:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.011+0000 7f05d1ffb700 1 -- 172.21.15.22:0/1309247531 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c8011990 con 0x7f05d4159870 2023-07-06T01:19:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.011+0000 7f05d1ffb700 1 -- 172.21.15.22:0/1309247531 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8011d20 con 0x7f05d4159870 2023-07-06T01:19:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.012+0000 7f05d9514700 1 -- 172.21.15.22:0/1309247531 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05d4159870 msgr2=0x7f05d415df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.012+0000 7f05d9514700 1 --2- 172.21.15.22:0/1309247531 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05d4159870 0x7f05d415df70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009840 tx=0x7f05c8009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.012+0000 7f05d9514700 1 -- 172.21.15.22:0/1309247531 shutdown_connections 2023-07-06T01:19:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.012+0000 7f05d9514700 1 --2- 172.21.15.22:0/1309247531 >> v2:172.21.15.103:3300/0 conn(0x7f05d4071a60 0x7f05d40ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.012+0000 7f05d9514700 1 --2- 172.21.15.22:0/1309247531 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05d4159870 0x7f05d415df70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.012+0000 7f05d9514700 1 -- 172.21.15.22:0/1309247531 >> 172.21.15.22:0/1309247531 conn(0x7f05d406b110 msgr2=0x7f05d406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:50.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.012+0000 7f05d9514700 1 -- 172.21.15.22:0/1309247531 shutdown_connections 2023-07-06T01:19:50.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.012+0000 7f05d9514700 1 -- 172.21.15.22:0/1309247531 wait complete. 2023-07-06T01:19:50.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.013+0000 7f05d9514700 1 Processor -- start 2023-07-06T01:19:50.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.013+0000 7f05d9514700 1 -- start start 2023-07-06T01:19:50.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d9514700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05d40715f0 0x7f05d415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d9514700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05d4071a60 0x7f05d41e2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d9514700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d4159870 0x7f05d4154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d9514700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d4153520 con 0x7f05d4159870 2023-07-06T01:19:50.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d9514700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d415b640 con 0x7f05d40715f0 2023-07-06T01:19:50.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d9514700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d415b920 con 0x7f05d4071a60 2023-07-06T01:19:50.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d4159870 0x7f05d4154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d4159870 0x7f05d4154760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46156/0 (socket says 172.21.15.22:46156) 2023-07-06T01:19:50.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d37fe700 1 -- 172.21.15.22:0/3589384844 learned_addr learned my addr 172.21.15.22:0/3589384844 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:50.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d27fc700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05d4071a60 0x7f05d41e2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05d40715f0 0x7f05d415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d37fe700 1 -- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05d4071a60 msgr2=0x7f05d41e2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d37fe700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05d4071a60 0x7f05d41e2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d37fe700 1 -- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05d40715f0 msgr2=0x7f05d415c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d37fe700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05d40715f0 0x7f05d415c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.014+0000 7f05d37fe700 1 -- 172.21.15.22:0/3589384844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc009a70 con 0x7f05d4159870 2023-07-06T01:19:50.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.015+0000 7f05d37fe700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d4159870 0x7f05d4154760 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00e580 tx=0x7f05cc00e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:50.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.015+0000 7f05bb7fe700 1 -- 172.21.15.22:0/3589384844 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc003c20 con 0x7f05d4159870 2023-07-06T01:19:50.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.015+0000 7f05bb7fe700 1 -- 172.21.15.22:0/3589384844 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05cc003d80 con 0x7f05d4159870 2023-07-06T01:19:50.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.015+0000 7f05d9514700 1 -- 172.21.15.22:0/3589384844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c80094a0 con 0x7f05d4159870 2023-07-06T01:19:50.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.015+0000 7f05bb7fe700 1 -- 172.21.15.22:0/3589384844 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc004260 con 0x7f05d4159870 2023-07-06T01:19:50.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.015+0000 7f05d9514700 1 -- 172.21.15.22:0/3589384844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d415bee0 con 0x7f05d4159870 2023-07-06T01:19:50.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.016+0000 7f05d9514700 1 -- 172.21.15.22:0/3589384844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d40744b0 con 0x7f05d4159870 2023-07-06T01:19:50.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.016+0000 7f05bb7fe700 1 -- 172.21.15.22:0/3589384844 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f05cc0043c0 con 0x7f05d4159870 2023-07-06T01:19:50.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.017+0000 7f05bb7fe700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f05c006ace0 0x7f05c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.017+0000 7f05bb7fe700 1 -- 172.21.15.22:0/3589384844 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f05cc016070 con 0x7f05d4159870 2023-07-06T01:19:50.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.017+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f05c006ace0 0x7f05c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.017+0000 7f05d2ffd700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f05c006ace0 0x7f05c006d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009fd0 tx=0x7f05c801d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:50.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.021+0000 7f05bb7fe700 1 -- 172.21.15.22:0/3589384844 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f05cc0572e0 con 0x7f05d4159870 2023-07-06T01:19:50.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.222+0000 7f05d9514700 1 -- 172.21.15.22:0/3589384844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05d406d010 con 0x7f05d4159870 2023-07-06T01:19:50.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.223+0000 7f05bb7fe700 1 -- 172.21.15.22:0/3589384844 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7f05cc05a900 con 0x7f05d4159870 2023-07-06T01:19:50.226 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:50.226 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:50.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.225+0000 7f05d9514700 1 -- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f05c006ace0 msgr2=0x7f05c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.225+0000 7f05d9514700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f05c006ace0 0x7f05c006d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009fd0 tx=0x7f05c801d040 comp rx=0 tx=0).stop 2023-07-06T01:19:50.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.225+0000 7f05d9514700 1 -- 172.21.15.22:0/3589384844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d4159870 msgr2=0x7f05d4154760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.225+0000 7f05d9514700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d4159870 0x7f05d4154760 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00e580 tx=0x7f05cc00e890 comp rx=0 tx=0).stop 2023-07-06T01:19:50.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.225+0000 7f05d9514700 1 -- 172.21.15.22:0/3589384844 shutdown_connections 2023-07-06T01:19:50.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.225+0000 7f05d9514700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05d4159870 0x7f05d4154760 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.225+0000 7f05d9514700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f05c006ace0 0x7f05c006d190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.225+0000 7f05d9514700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05d40715f0 0x7f05d415c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.225+0000 7f05d9514700 1 --2- 172.21.15.22:0/3589384844 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05d4071a60 0x7f05d41e2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.225+0000 7f05d9514700 1 -- 172.21.15.22:0/3589384844 >> 172.21.15.22:0/3589384844 conn(0x7f05d406b110 msgr2=0x7f05d40b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:50.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.225+0000 7f05d9514700 1 -- 172.21.15.22:0/3589384844 shutdown_connections 2023-07-06T01:19:50.230 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.225+0000 7f05d9514700 1 -- 172.21.15.22:0/3589384844 wait complete. 2023-07-06T01:19:50.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T01:19:22.873255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T01:19:24.019088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:19:50.242 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:19:50.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.367+0000 7fba76c04700 1 Processor -- start 2023-07-06T01:19:50.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.367+0000 7fba76c04700 1 -- start start 2023-07-06T01:19:50.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.368+0000 7fba76c04700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fba70159360 0x7fba7015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.368+0000 7fba76c04700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba700cf230 con 0x7fba700ccae0 2023-07-06T01:19:50.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.368+0000 7fba76c04700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba70071840 con 0x7fba700cc660 2023-07-06T01:19:50.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.368+0000 7fba76c04700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7015df20 con 0x7fba70159360 2023-07-06T01:19:50.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.368+0000 7fba749a0700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fba70159360 0x7fba7015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.368+0000 7fba751a1700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fba700ccae0 0x7fba70071130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36904/0 (socket says 172.21.15.22:36904) 2023-07-06T01:19:50.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.368+0000 7fba751a1700 1 -- 172.21.15.22:0/369941612 learned_addr learned my addr 172.21.15.22:0/369941612 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:50.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.368+0000 7fba749a0700 1 -- 172.21.15.22:0/369941612 >> v1:172.21.15.103:6789/0 conn(0x7fba700cc660 legacy=0x7fba70070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.368+0000 7fba749a0700 1 -- 172.21.15.22:0/369941612 >> v1:172.21.15.22:6789/0 conn(0x7fba700ccae0 legacy=0x7fba70071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.369+0000 7fba749a0700 1 -- 172.21.15.22:0/369941612 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba700ce620 con 0x7fba70159360 2023-07-06T01:19:50.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.368+0000 7fba6f7fe700 1 -- 172.21.15.22:0/369941612 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190301301 0 0) 0x7fba700cf230 con 0x7fba700ccae0 2023-07-06T01:19:50.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.369+0000 7fba749a0700 1 --2- 172.21.15.22:0/369941612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba70159360 0x7fba7015d9e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fba64009840 tx=0x7fba64009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5576c9a77d885f2e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:50.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.369+0000 7fba6f7fe700 1 -- 172.21.15.22:0/369941612 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64011830 con 0x7fba70159360 2023-07-06T01:19:50.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.369+0000 7fba6f7fe700 1 -- 172.21.15.22:0/369941612 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba64011990 con 0x7fba70159360 2023-07-06T01:19:50.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.369+0000 7fba6f7fe700 1 -- 172.21.15.22:0/369941612 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64011d20 con 0x7fba70159360 2023-07-06T01:19:50.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.369+0000 7fba76c04700 1 -- 172.21.15.22:0/369941612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba70159360 msgr2=0x7fba7015d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.369+0000 7fba76c04700 1 --2- 172.21.15.22:0/369941612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba70159360 0x7fba7015d9e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fba64009840 tx=0x7fba64009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:50.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.369+0000 7fba76c04700 1 -- 172.21.15.22:0/369941612 shutdown_connections 2023-07-06T01:19:50.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.369+0000 7fba76c04700 1 --2- 172.21.15.22:0/369941612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba70159360 0x7fba7015d9e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.369+0000 7fba76c04700 1 -- 172.21.15.22:0/369941612 >> 172.21.15.22:0/369941612 conn(0x7fba700b1890 msgr2=0x7fba700b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:50.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.369+0000 7fba76c04700 1 -- 172.21.15.22:0/369941612 shutdown_connections 2023-07-06T01:19:50.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.369+0000 7fba76c04700 1 -- 172.21.15.22:0/369941612 wait complete. 2023-07-06T01:19:50.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.371+0000 7fba76c04700 1 Processor -- start 2023-07-06T01:19:50.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.371+0000 7fba76c04700 1 -- start start 2023-07-06T01:19:50.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.371+0000 7fba76c04700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba700cc660 0x7fba701502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.371+0000 7fba76c04700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba700ccae0 0x7fba7006a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.371+0000 7fba76c04700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fba70159360 0x7fba7006a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.371+0000 7fba76c04700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba700ae8f0 con 0x7fba700cc660 2023-07-06T01:19:50.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.371+0000 7fba76c04700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba700aebd0 con 0x7fba700ccae0 2023-07-06T01:19:50.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.371+0000 7fba76c04700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba700aeeb0 con 0x7fba70159360 2023-07-06T01:19:50.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba749a0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba700cc660 0x7fba701502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba749a0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba700cc660 0x7fba701502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46164/0 (socket says 172.21.15.22:46164) 2023-07-06T01:19:50.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba749a0700 1 -- 172.21.15.22:0/323375308 learned_addr learned my addr 172.21.15.22:0/323375308 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba6ffff700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba700ccae0 0x7fba7006a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba751a1700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fba70159360 0x7fba7006a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba749a0700 1 -- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fba70159360 msgr2=0x7fba7006a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba749a0700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fba70159360 0x7fba7006a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba749a0700 1 -- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba700ccae0 msgr2=0x7fba7006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba749a0700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba700ccae0 0x7fba7006a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba749a0700 1 -- 172.21.15.22:0/323375308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba640094a0 con 0x7fba700cc660 2023-07-06T01:19:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba749a0700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba700cc660 0x7fba701502d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba64014040 tx=0x7fba64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba6d7fa700 1 -- 172.21.15.22:0/323375308 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6401a900 con 0x7fba700cc660 2023-07-06T01:19:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba6d7fa700 1 -- 172.21.15.22:0/323375308 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6401aa60 con 0x7fba700cc660 2023-07-06T01:19:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba76c04700 1 -- 172.21.15.22:0/323375308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba700af190 con 0x7fba700cc660 2023-07-06T01:19:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba6d7fa700 1 -- 172.21.15.22:0/323375308 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6401add0 con 0x7fba700cc660 2023-07-06T01:19:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.372+0000 7fba76c04700 1 -- 172.21.15.22:0/323375308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba70150810 con 0x7fba700cc660 2023-07-06T01:19:50.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.373+0000 7fba76c04700 1 -- 172.21.15.22:0/323375308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba700744b0 con 0x7fba700cc660 2023-07-06T01:19:50.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.374+0000 7fba6d7fa700 1 -- 172.21.15.22:0/323375308 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba64022070 con 0x7fba700cc660 2023-07-06T01:19:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.374+0000 7fba6d7fa700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fba5406ace0 0x7fba5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.374+0000 7fba6d7fa700 1 -- 172.21.15.22:0/323375308 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba6408a6e0 con 0x7fba700cc660 2023-07-06T01:19:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.374+0000 7fba6ffff700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fba5406ace0 0x7fba5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.375+0000 7fba6ffff700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fba5406ace0 0x7fba5406d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba701f5e70 tx=0x7fba5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:50.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.378+0000 7fba6d7fa700 1 -- 172.21.15.22:0/323375308 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fba64055820 con 0x7fba700cc660 2023-07-06T01:19:50.579 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.577+0000 7fba76c04700 1 -- 172.21.15.22:0/323375308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fba700cddc0 con 0x7fba700cc660 2023-07-06T01:19:50.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.577+0000 7fba6d7fa700 1 -- 172.21.15.22:0/323375308 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fba64058e40 con 0x7fba700cc660 2023-07-06T01:19:50.580 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:19:50.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.579+0000 7fba76c04700 1 -- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fba5406ace0 msgr2=0x7fba5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.580+0000 7fba76c04700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fba5406ace0 0x7fba5406d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba701f5e70 tx=0x7fba5c009040 comp rx=0 tx=0).stop 2023-07-06T01:19:50.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.580+0000 7fba76c04700 1 -- 172.21.15.22:0/323375308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba700cc660 msgr2=0x7fba701502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.580+0000 7fba76c04700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba700cc660 0x7fba701502d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba64014040 tx=0x7fba64004750 comp rx=0 tx=0).stop 2023-07-06T01:19:50.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.580+0000 7fba76c04700 1 -- 172.21.15.22:0/323375308 shutdown_connections 2023-07-06T01:19:50.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.580+0000 7fba76c04700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fba70159360 0x7fba7006a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.580+0000 7fba76c04700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fba700cc660 0x7fba701502d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.580+0000 7fba76c04700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fba5406ace0 0x7fba5406d190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.580+0000 7fba76c04700 1 --2- 172.21.15.22:0/323375308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba700ccae0 0x7fba7006a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.580+0000 7fba76c04700 1 -- 172.21.15.22:0/323375308 >> 172.21.15.22:0/323375308 conn(0x7fba700b1890 msgr2=0x7fba700afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:50.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.580+0000 7fba76c04700 1 -- 172.21.15.22:0/323375308 shutdown_connections 2023-07-06T01:19:50.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.580+0000 7fba76c04700 1 -- 172.21.15.22:0/323375308 wait complete. 2023-07-06T01:19:50.595 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:19:50.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.719+0000 7f85efb40700 1 Processor -- start 2023-07-06T01:19:50.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.720+0000 7f85efb40700 1 -- start start 2023-07-06T01:19:50.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.720+0000 7f85efb40700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e815a610 0x7f85e806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.720+0000 7f85efb40700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f85e815eae0 0x7f85e806c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.720+0000 7f85efb40700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f85e806c730 0x7f85e806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.720+0000 7f85efb40700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e815a9e0 con 0x7f85e815eae0 2023-07-06T01:19:50.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85efb40700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e806cbb0 con 0x7f85e806c730 2023-07-06T01:19:50.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85efb40700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e8071f30 con 0x7f85e815a610 2023-07-06T01:19:50.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ed0db700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f85e815eae0 0x7f85e806c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ed0db700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f85e815eae0 0x7f85e806c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46174/0 (socket says 172.21.15.22:46174) 2023-07-06T01:19:50.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ed0db700 1 -- 172.21.15.22:0/1791704091 learned_addr learned my addr 172.21.15.22:0/1791704091 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:50.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ee0dd700 1 --2- 172.21.15.22:0/1791704091 >> v2:172.21.15.103:3300/0 conn(0x7f85e806c730 0x7f85e806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ed8dc700 1 --2- 172.21.15.22:0/1791704091 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e815a610 0x7f85e806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ed0db700 1 -- 172.21.15.22:0/1791704091 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e815a610 msgr2=0x7f85e806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:50.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ed0db700 1 --2- 172.21.15.22:0/1791704091 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e815a610 0x7f85e806b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ed0db700 1 -- 172.21.15.22:0/1791704091 >> v2:172.21.15.103:3300/0 conn(0x7f85e806c730 msgr2=0x7f85e806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:50.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ed0db700 1 --2- 172.21.15.22:0/1791704091 >> v2:172.21.15.103:3300/0 conn(0x7f85e806c730 0x7f85e806a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ed0db700 1 -- 172.21.15.22:0/1791704091 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e8071320 con 0x7f85e815eae0 2023-07-06T01:19:50.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ed8dc700 1 --2- 172.21.15.22:0/1791704091 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e815a610 0x7f85e806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:50.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ee0dd700 1 --2- 172.21.15.22:0/1791704091 >> v2:172.21.15.103:3300/0 conn(0x7f85e806c730 0x7f85e806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:50.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ed0db700 1 --2- 172.21.15.22:0/1791704091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e815eae0 0x7f85e806c1f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f85e80d0340 tx=0x7f85e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e65001b7874485 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:50.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ec8da700 1 -- 172.21.15.22:0/1791704091 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e400b870 con 0x7f85e815eae0 2023-07-06T01:19:50.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ec8da700 1 -- 172.21.15.22:0/1791704091 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e400b9d0 con 0x7f85e815eae0 2023-07-06T01:19:50.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.721+0000 7f85ec8da700 1 -- 172.21.15.22:0/1791704091 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e400bd60 con 0x7f85e815eae0 2023-07-06T01:19:50.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.722+0000 7f85efb40700 1 -- 172.21.15.22:0/1791704091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e815eae0 msgr2=0x7f85e806c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.722+0000 7f85efb40700 1 --2- 172.21.15.22:0/1791704091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e815eae0 0x7f85e806c1f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f85e80d0340 tx=0x7f85e4009bd0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.722+0000 7f85efb40700 1 -- 172.21.15.22:0/1791704091 shutdown_connections 2023-07-06T01:19:50.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.722+0000 7f85efb40700 1 --2- 172.21.15.22:0/1791704091 >> v2:172.21.15.103:3300/0 conn(0x7f85e806c730 0x7f85e806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.722+0000 7f85efb40700 1 --2- 172.21.15.22:0/1791704091 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e815a610 0x7f85e806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.722+0000 7f85efb40700 1 --2- 172.21.15.22:0/1791704091 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e815eae0 0x7f85e806c1f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.722+0000 7f85efb40700 1 -- 172.21.15.22:0/1791704091 >> 172.21.15.22:0/1791704091 conn(0x7f85e80ccb00 msgr2=0x7f85e80cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:50.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.722+0000 7f85efb40700 1 -- 172.21.15.22:0/1791704091 shutdown_connections 2023-07-06T01:19:50.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.722+0000 7f85efb40700 1 -- 172.21.15.22:0/1791704091 wait complete. 2023-07-06T01:19:50.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.723+0000 7f85efb40700 1 Processor -- start 2023-07-06T01:19:50.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.723+0000 7f85efb40700 1 -- start start 2023-07-06T01:19:50.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85efb40700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e806c730 0x7f85e80b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85efb40700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e815a610 0x7f85e81418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85efb40700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85e815eae0 0x7f85e8141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85efb40700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e8157fd0 con 0x7f85e806c730 2023-07-06T01:19:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85efb40700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e8158870 con 0x7f85e815eae0 2023-07-06T01:19:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85efb40700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80b5290 con 0x7f85e815a610 2023-07-06T01:19:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85ed8dc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e806c730 0x7f85e80b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85ed8dc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e806c730 0x7f85e80b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46178/0 (socket says 172.21.15.22:46178) 2023-07-06T01:19:50.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85ed8dc700 1 -- 172.21.15.22:0/1484766270 learned_addr learned my addr 172.21.15.22:0/1484766270 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:50.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85ee0dd700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85e815eae0 0x7f85e8141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85ed0db700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e815a610 0x7f85e81418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85ed8dc700 1 -- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e815a610 msgr2=0x7f85e81418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85ed8dc700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e815a610 0x7f85e81418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85ed8dc700 1 -- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85e815eae0 msgr2=0x7f85e8141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85ed8dc700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85e815eae0 0x7f85e8141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85ed8dc700 1 -- 172.21.15.22:0/1484766270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e4009550 con 0x7f85e806c730 2023-07-06T01:19:50.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85ee0dd700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85e815eae0 0x7f85e8141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:50.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.724+0000 7f85ed0db700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e815a610 0x7f85e81418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:50.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.725+0000 7f85ed8dc700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e806c730 0x7f85e80b2220 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00e810 tx=0x7f85dc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:50.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.725+0000 7f85da7fc700 1 -- 172.21.15.22:0/1484766270 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc004230 con 0x7f85e806c730 2023-07-06T01:19:50.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.725+0000 7f85da7fc700 1 -- 172.21.15.22:0/1484766270 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc004390 con 0x7f85e806c730 2023-07-06T01:19:50.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.725+0000 7f85efb40700 1 -- 172.21.15.22:0/1484766270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e813f580 con 0x7f85e806c730 2023-07-06T01:19:50.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.725+0000 7f85da7fc700 1 -- 172.21.15.22:0/1484766270 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0036a0 con 0x7f85e806c730 2023-07-06T01:19:50.735 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.725+0000 7f85efb40700 1 -- 172.21.15.22:0/1484766270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e81581b0 con 0x7f85e806c730 2023-07-06T01:19:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.726+0000 7f85efb40700 1 -- 172.21.15.22:0/1484766270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e8158ae0 con 0x7f85e806c730 2023-07-06T01:19:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.726+0000 7f85da7fc700 1 -- 172.21.15.22:0/1484766270 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f85dc003800 con 0x7f85e806c730 2023-07-06T01:19:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.727+0000 7f85da7fc700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f85cc06ace0 0x7f85cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.727+0000 7f85da7fc700 1 -- 172.21.15.22:0/1484766270 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f85dc088e80 con 0x7f85e806c730 2023-07-06T01:19:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.727+0000 7f85ed0db700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f85cc06ace0 0x7f85cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:50.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.727+0000 7f85ed0db700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f85cc06ace0 0x7f85cc06d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f85e4014040 tx=0x7f85e4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:50.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.731+0000 7f85da7fc700 1 -- 172.21.15.22:0/1484766270 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f85dc053fc0 con 0x7f85e806c730 2023-07-06T01:19:50.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.930+0000 7f85efb40700 1 -- 172.21.15.22:0/1484766270 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85e80c6210 con 0x7f85e806c730 2023-07-06T01:19:50.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.930+0000 7f85da7fc700 1 -- 172.21.15.22:0/1484766270 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85dc0575e0 con 0x7f85e806c730 2023-07-06T01:19:50.933 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:19:50.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.932+0000 7f85efb40700 1 -- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f85cc06ace0 msgr2=0x7f85cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.932+0000 7f85efb40700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f85cc06ace0 0x7f85cc06d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f85e4014040 tx=0x7f85e4008030 comp rx=0 tx=0).stop 2023-07-06T01:19:50.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.932+0000 7f85efb40700 1 -- 172.21.15.22:0/1484766270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e806c730 msgr2=0x7f85e80b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:50.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.932+0000 7f85efb40700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e806c730 0x7f85e80b2220 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00e810 tx=0x7f85dc00ebd0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.932+0000 7f85efb40700 1 -- 172.21.15.22:0/1484766270 shutdown_connections 2023-07-06T01:19:50.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.932+0000 7f85efb40700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f85cc06ace0 0x7f85cc06d190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.932+0000 7f85efb40700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85e815eae0 0x7f85e8141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.932+0000 7f85efb40700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85e806c730 0x7f85e80b2220 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.932+0000 7f85efb40700 1 --2- 172.21.15.22:0/1484766270 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e815a610 0x7f85e81418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:50.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.932+0000 7f85efb40700 1 -- 172.21.15.22:0/1484766270 >> 172.21.15.22:0/1484766270 conn(0x7f85e80ccb00 msgr2=0x7f85e80b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.933+0000 7f85efb40700 1 -- 172.21.15.22:0/1484766270 shutdown_connections 2023-07-06T01:19:50.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:50.933+0000 7f85efb40700 1 -- 172.21.15.22:0/1484766270 wait complete. 2023-07-06T01:19:50.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T01:19:51.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.072+0000 7f4490c0a700 1 Processor -- start 2023-07-06T01:19:51.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.072+0000 7f4490c0a700 1 -- start start 2023-07-06T01:19:51.077 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.073+0000 7f4490c0a700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f448c0af2a0 0x7f448c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.073+0000 7f4490c0a700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f448c06a000 0x7f448c06a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.073+0000 7f4490c0a700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448c0af670 con 0x7f448c0cbc10 2023-07-06T01:19:51.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.073+0000 7f4490c0a700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448c06a9e0 con 0x7f448c06a000 2023-07-06T01:19:51.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.073+0000 7f4490c0a700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448c0cd560 con 0x7f448c0af2a0 2023-07-06T01:19:51.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.073+0000 7f4489d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f448c06a000 0x7f448c06a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.073+0000 7f4489d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f448c06a000 0x7f448c06a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46186/0 (socket says 172.21.15.22:46186) 2023-07-06T01:19:51.078 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.073+0000 7f448ad9d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f448c0cbc10 0x7f448c0cc050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36932/0 (socket says 172.21.15.22:36932) 2023-07-06T01:19:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.073+0000 7f4489d9b700 1 -- 172.21.15.22:0/3266947522 learned_addr learned my addr 172.21.15.22:0/3266947522 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.073+0000 7f448a59c700 1 --2- 172.21.15.22:0/3266947522 >> v2:172.21.15.103:3300/0 conn(0x7f448c0af2a0 0x7f448c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.073+0000 7f4489d9b700 1 -- 172.21.15.22:0/3266947522 >> v2:172.21.15.103:3300/0 conn(0x7f448c0af2a0 msgr2=0x7f448c06caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.073+0000 7f4489d9b700 1 --2- 172.21.15.22:0/3266947522 >> v2:172.21.15.103:3300/0 conn(0x7f448c0af2a0 0x7f448c06caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.073+0000 7f4489d9b700 1 -- 172.21.15.22:0/3266947522 >> v1:172.21.15.22:6789/0 conn(0x7f448c0cbc10 legacy=0x7f448c0cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.074+0000 7f448959a700 1 -- 172.21.15.22:0/3266947522 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037080918 0 0) 0x7f448c0af670 con 0x7f448c0cbc10 2023-07-06T01:19:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.074+0000 7f4489d9b700 1 -- 172.21.15.22:0/3266947522 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c06d030 con 0x7f448c06a000 2023-07-06T01:19:51.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.074+0000 7f4489d9b700 1 --2- 172.21.15.22:0/3266947522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f448c06a000 0x7f448c06a4a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f448c067460 tx=0x7f447c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31b755e13027034 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.074+0000 7f448959a700 1 -- 172.21.15.22:0/3266947522 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c00b870 con 0x7f448c06a000 2023-07-06T01:19:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.074+0000 7f448959a700 1 -- 172.21.15.22:0/3266947522 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c00b9d0 con 0x7f448c06a000 2023-07-06T01:19:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.074+0000 7f448959a700 1 -- 172.21.15.22:0/3266947522 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c00bd60 con 0x7f448c06a000 2023-07-06T01:19:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.074+0000 7f4490c0a700 1 -- 172.21.15.22:0/3266947522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f448c06a000 msgr2=0x7f448c06a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.074+0000 7f4490c0a700 1 --2- 172.21.15.22:0/3266947522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f448c06a000 0x7f448c06a4a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f448c067460 tx=0x7f447c009bd0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.074+0000 7f4490c0a700 1 -- 172.21.15.22:0/3266947522 shutdown_connections 2023-07-06T01:19:51.080 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.074+0000 7f4490c0a700 1 --2- 172.21.15.22:0/3266947522 >> v2:172.21.15.103:3300/0 conn(0x7f448c0af2a0 0x7f448c06caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.074+0000 7f4490c0a700 1 --2- 172.21.15.22:0/3266947522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f448c06a000 0x7f448c06a4a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.074+0000 7f4490c0a700 1 -- 172.21.15.22:0/3266947522 >> 172.21.15.22:0/3266947522 conn(0x7f448c0b1890 msgr2=0x7f448c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:51.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.075+0000 7f4490c0a700 1 -- 172.21.15.22:0/3266947522 shutdown_connections 2023-07-06T01:19:51.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.075+0000 7f4490c0a700 1 -- 172.21.15.22:0/3266947522 wait complete. 2023-07-06T01:19:51.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.076+0000 7f4490c0a700 1 Processor -- start 2023-07-06T01:19:51.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.076+0000 7f4490c0a700 1 -- start start 2023-07-06T01:19:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f4490c0a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f448c06a000 0x7f448c159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f4490c0a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f448c0af2a0 0x7f448c1f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f4490c0a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f448c0cbc10 0x7f448c0c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f4490c0a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c0c33f0 con 0x7f448c06a000 2023-07-06T01:19:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f4490c0a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c1f3220 con 0x7f448c0af2a0 2023-07-06T01:19:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f4490c0a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c159e30 con 0x7f448c0cbc10 2023-07-06T01:19:51.084 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f448a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f448c06a000 0x7f448c159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f448a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f448c06a000 0x7f448c159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46190/0 (socket says 172.21.15.22:46190) 2023-07-06T01:19:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f448a59c700 1 -- 172.21.15.22:0/3529210832 learned_addr learned my addr 172.21.15.22:0/3529210832 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f4489d9b700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f448c0af2a0 0x7f448c1f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f448ad9d700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f448c0cbc10 0x7f448c0c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f448a59c700 1 -- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f448c0cbc10 msgr2=0x7f448c0c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f448a59c700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f448c0cbc10 0x7f448c0c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f448a59c700 1 -- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f448c0af2a0 msgr2=0x7f448c1f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f448a59c700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f448c0af2a0 0x7f448c1f2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f448a59c700 1 -- 172.21.15.22:0/3529210832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c009550 con 0x7f448c06a000 2023-07-06T01:19:51.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f4489d9b700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f448c0af2a0 0x7f448c1f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:51.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f448a59c700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f448c06a000 0x7f448c159850 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f448000e840 tx=0x7f448000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:51.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f447affd700 1 -- 172.21.15.22:0/3529210832 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480013610 con 0x7f448c06a000 2023-07-06T01:19:51.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f447affd700 1 -- 172.21.15.22:0/3529210832 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4480013770 con 0x7f448c06a000 2023-07-06T01:19:51.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f4490c0a700 1 -- 172.21.15.22:0/3529210832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448c0c3640 con 0x7f448c06a000 2023-07-06T01:19:51.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.077+0000 7f447affd700 1 -- 172.21.15.22:0/3529210832 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480013b20 con 0x7f448c06a000 2023-07-06T01:19:51.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.078+0000 7f4490c0a700 1 -- 172.21.15.22:0/3529210832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c0c3c00 con 0x7f448c06a000 2023-07-06T01:19:51.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.078+0000 7f4490c0a700 1 -- 172.21.15.22:0/3529210832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448c15ca90 con 0x7f448c06a000 2023-07-06T01:19:51.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.079+0000 7f447affd700 1 -- 172.21.15.22:0/3529210832 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4480013c80 con 0x7f448c06a000 2023-07-06T01:19:51.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.079+0000 7f447affd700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f446c06ace0 0x7f446c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.079+0000 7f447affd700 1 -- 172.21.15.22:0/3529210832 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f44800890e0 con 0x7f448c06a000 2023-07-06T01:19:51.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.080+0000 7f4489d9b700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f446c06ace0 0x7f446c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.080+0000 7f4489d9b700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f446c06ace0 0x7f446c06d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f447c000c00 tx=0x7f447c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:51.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.084+0000 7f447affd700 1 -- 172.21.15.22:0/3529210832 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f448c15ca90 con 0x7f448c06a000 2023-07-06T01:19:51.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.281+0000 7f4490c0a700 1 -- 172.21.15.22:0/3529210832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f448c15ca90 con 0x7f448c06a000 2023-07-06T01:19:51.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.281+0000 7f447affd700 1 -- 172.21.15.22:0/3529210832 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f448c15ca90 con 0x7f448c06a000 2023-07-06T01:19:51.284 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-07-06T01:19:51.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.283+0000 7f4490c0a700 1 -- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f446c06ace0 msgr2=0x7f446c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.283+0000 7f4490c0a700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f446c06ace0 0x7f446c06d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f447c000c00 tx=0x7f447c008030 comp rx=0 tx=0).stop 2023-07-06T01:19:51.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.283+0000 7f4490c0a700 1 -- 172.21.15.22:0/3529210832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f448c06a000 msgr2=0x7f448c159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.283+0000 7f4490c0a700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f448c06a000 0x7f448c159850 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f448000e840 tx=0x7f448000ec00 comp rx=0 tx=0).stop 2023-07-06T01:19:51.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.283+0000 7f4490c0a700 1 -- 172.21.15.22:0/3529210832 shutdown_connections 2023-07-06T01:19:51.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.283+0000 7f4490c0a700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f448c0cbc10 0x7f448c0c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.283+0000 7f4490c0a700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f448c06a000 0x7f448c159850 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.283+0000 7f4490c0a700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f446c06ace0 0x7f446c06d190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.283+0000 7f4490c0a700 1 --2- 172.21.15.22:0/3529210832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f448c0af2a0 0x7f448c1f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.283+0000 7f4490c0a700 1 -- 172.21.15.22:0/3529210832 >> 172.21.15.22:0/3529210832 conn(0x7f448c0b1890 msgr2=0x7f448c0c6630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:51.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.284+0000 7f4490c0a700 1 -- 172.21.15.22:0/3529210832 shutdown_connections 2023-07-06T01:19:51.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.284+0000 7f4490c0a700 1 -- 172.21.15.22:0/3529210832 wait complete. 2023-07-06T01:19:51.298 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T01:19:51.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.423+0000 7fe02cb6f700 1 Processor -- start 2023-07-06T01:19:51.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.423+0000 7fe02cb6f700 1 -- start start 2023-07-06T01:19:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.424+0000 7fe02cb6f700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe028158bb0 0x7fe02806b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.424+0000 7fe02cb6f700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe028158f80 con 0x7fe02815c6b0 2023-07-06T01:19:51.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.424+0000 7fe02cb6f700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02815ce90 con 0x7fe02806c900 2023-07-06T01:19:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.424+0000 7fe02cb6f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280701e0 con 0x7fe028158bb0 2023-07-06T01:19:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.424+0000 7fe025d9b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe02815c6b0 0x7fe02806c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36942/0 (socket says 172.21.15.22:36942) 2023-07-06T01:19:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.424+0000 7fe025d9b700 1 -- 172.21.15.22:0/2939120266 learned_addr learned my addr 172.21.15.22:0/2939120266 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.424+0000 7fe02659c700 1 --2- 172.21.15.22:0/2939120266 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe028158bb0 0x7fe02806b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.424+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561741518 0 0) 0x7fe028158f80 con 0x7fe02815c6b0 2023-07-06T01:19:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.424+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0100035b0 con 0x7fe02815c6b0 2023-07-06T01:19:51.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152424807 0 0) 0x7fe02815ce90 con 0x7fe02806c900 2023-07-06T01:19:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe028158f80 con 0x7fe02806c900 2023-07-06T01:19:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4125681824 0 0) 0x7fe0100035b0 con 0x7fe02815c6b0 2023-07-06T01:19:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe028158bb0 msgr2=0x7fe02806b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 --2- 172.21.15.22:0/2939120266 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe028158bb0 0x7fe02806b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 >> v1:172.21.15.103:6789/0 conn(0x7fe02806c900 legacy=0x7fe028070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02815d170 con 0x7fe02815c6b0 2023-07-06T01:19:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7fe018001670 con 0x7fe02815c6b0 2023-07-06T01:19:51.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02659c700 1 --2- 172.21.15.22:0/2939120266 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe028158bb0 0x7fe02806b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 >> v1:172.21.15.22:6789/0 conn(0x7fe02815c6b0 legacy=0x7fe02806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 --2- 172.21.15.22:0/2939120266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe010003bc0 0x7fe010003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 --2- 172.21.15.22:0/2939120266 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe028158bb0 0x7fe010004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 --2- 172.21.15.22:0/2939120266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe010004d90 0x7fe010005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280701e0 con 0x7fe010004d90 2023-07-06T01:19:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01000bac0 con 0x7fe010003bc0 2023-07-06T01:19:51.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01000bc00 con 0x7fe028158bb0 2023-07-06T01:19:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe026d9d700 1 --2- 172.21.15.22:0/2939120266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe010004d90 0x7fe010005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02659c700 1 --2- 172.21.15.22:0/2939120266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe010003bc0 0x7fe010003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe025d9b700 1 --2- 172.21.15.22:0/2939120266 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe028158bb0 0x7fe010004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe026d9d700 1 -- 172.21.15.22:0/2939120266 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe028158bb0 msgr2=0x7fe010004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe026d9d700 1 --2- 172.21.15.22:0/2939120266 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe028158bb0 0x7fe010004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe026d9d700 1 -- 172.21.15.22:0/2939120266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe010003bc0 msgr2=0x7fe010003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:51.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe026d9d700 1 --2- 172.21.15.22:0/2939120266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe010003bc0 0x7fe010003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe026d9d700 1 -- 172.21.15.22:0/2939120266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c0094a0 con 0x7fe010004d90 2023-07-06T01:19:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe026d9d700 1 --2- 172.21.15.22:0/2939120266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe010004d90 0x7fe010005200 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe020002230 tx=0x7fe010005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a874b3bae9b75f0a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.425+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020001f30 con 0x7fe010004d90 2023-07-06T01:19:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.426+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02000ce60 con 0x7fe010004d90 2023-07-06T01:19:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.426+0000 7fe02559a700 1 -- 172.21.15.22:0/2939120266 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020003c40 con 0x7fe010004d90 2023-07-06T01:19:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.426+0000 7fe02cb6f700 1 -- 172.21.15.22:0/2939120266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe010004d90 msgr2=0x7fe010005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.426+0000 7fe02cb6f700 1 --2- 172.21.15.22:0/2939120266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe010004d90 0x7fe010005200 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe020002230 tx=0x7fe010005880 comp rx=0 tx=0).stop 2023-07-06T01:19:51.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.426+0000 7fe026d9d700 1 -- 172.21.15.22:0/2939120266 reap_dead start 2023-07-06T01:19:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.426+0000 7fe02cb6f700 1 -- 172.21.15.22:0/2939120266 shutdown_connections 2023-07-06T01:19:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.426+0000 7fe02cb6f700 1 -- 172.21.15.22:0/2939120266 >> 172.21.15.22:0/2939120266 conn(0x7fe0280ccb00 msgr2=0x7fe0280cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.426+0000 7fe02cb6f700 1 -- 172.21.15.22:0/2939120266 shutdown_connections 2023-07-06T01:19:51.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.426+0000 7fe02cb6f700 1 -- 172.21.15.22:0/2939120266 wait complete. 2023-07-06T01:19:51.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.428+0000 7fe02cb6f700 1 Processor -- start 2023-07-06T01:19:51.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.428+0000 7fe02cb6f700 1 -- start start 2023-07-06T01:19:51.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.428+0000 7fe02cb6f700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe010004d90 0x7fe0280b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.428+0000 7fe02cb6f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe02814b830 0x7fe0280cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.428+0000 7fe02cb6f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe028166a70 0x7fe028166ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.428+0000 7fe02cb6f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280b3fc0 con 0x7fe028166a70 2023-07-06T01:19:51.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.428+0000 7fe02cb6f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280b4b40 con 0x7fe02814b830 2023-07-06T01:19:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.428+0000 7fe02cb6f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0280b4210 con 0x7fe010004d90 2023-07-06T01:19:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.428+0000 7fe026d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe028166a70 0x7fe028166ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.428+0000 7fe026d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe028166a70 0x7fe028166ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46212/0 (socket says 172.21.15.22:46212) 2023-07-06T01:19:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.428+0000 7fe026d9d700 1 -- 172.21.15.22:0/631501495 learned_addr learned my addr 172.21.15.22:0/631501495 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe025d9b700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe02814b830 0x7fe0280cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe02659c700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe010004d90 0x7fe0280b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe026d9d700 1 -- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe010004d90 msgr2=0x7fe0280b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe026d9d700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe010004d90 0x7fe0280b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe026d9d700 1 -- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe02814b830 msgr2=0x7fe0280cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe026d9d700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe02814b830 0x7fe0280cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe026d9d700 1 -- 172.21.15.22:0/631501495 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0280b44f0 con 0x7fe028166a70 2023-07-06T01:19:51.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe02659c700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe010004d90 0x7fe0280b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:51.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe025d9b700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe02814b830 0x7fe0280cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:51.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe026d9d700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe028166a70 0x7fe028166ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe02000d010 tx=0x7fe020001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:51.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe016ffd700 1 -- 172.21.15.22:0/631501495 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0200050d0 con 0x7fe028166a70 2023-07-06T01:19:51.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe016ffd700 1 -- 172.21.15.22:0/631501495 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe020005250 con 0x7fe028166a70 2023-07-06T01:19:51.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe016ffd700 1 -- 172.21.15.22:0/631501495 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020005600 con 0x7fe028166a70 2023-07-06T01:19:51.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe02cb6f700 1 -- 172.21.15.22:0/631501495 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0280b47d0 con 0x7fe028166a70 2023-07-06T01:19:51.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.429+0000 7fe02cb6f700 1 -- 172.21.15.22:0/631501495 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0280b4fd0 con 0x7fe028166a70 2023-07-06T01:19:51.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.430+0000 7fe02cb6f700 1 -- 172.21.15.22:0/631501495 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0280b4990 con 0x7fe028166a70 2023-07-06T01:19:51.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.430+0000 7fe016ffd700 1 -- 172.21.15.22:0/631501495 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe020019070 con 0x7fe028166a70 2023-07-06T01:19:51.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.431+0000 7fe016ffd700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe00806ace0 0x7fe00806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.431+0000 7fe016ffd700 1 -- 172.21.15.22:0/631501495 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe02008a4d0 con 0x7fe028166a70 2023-07-06T01:19:51.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.431+0000 7fe02659c700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe00806ace0 0x7fe00806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.432+0000 7fe02659c700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe00806ace0 0x7fe00806d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe028146680 tx=0x7fe01c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:51.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.436+0000 7fe016ffd700 1 -- 172.21.15.22:0/631501495 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe020055610 con 0x7fe028166a70 2023-07-06T01:19:51.639 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.636+0000 7fe02cb6f700 1 -- 172.21.15.22:0/631501495 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0280b4cb0 con 0x7fe028166a70 2023-07-06T01:19:51.640 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.636+0000 7fe016ffd700 1 -- 172.21.15.22:0/631501495 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0200011c0 con 0x7fe028166a70 2023-07-06T01:19:51.640 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-07-06T01:19:51.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.638+0000 7fe02cb6f700 1 -- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe00806ace0 msgr2=0x7fe00806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.638+0000 7fe02cb6f700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe00806ace0 0x7fe00806d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe028146680 tx=0x7fe01c008040 comp rx=0 tx=0).stop 2023-07-06T01:19:51.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.638+0000 7fe02cb6f700 1 -- 172.21.15.22:0/631501495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe028166a70 msgr2=0x7fe028166ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.638+0000 7fe02cb6f700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe028166a70 0x7fe028166ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe02000d010 tx=0x7fe020001650 comp rx=0 tx=0).stop 2023-07-06T01:19:51.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.638+0000 7fe02cb6f700 1 -- 172.21.15.22:0/631501495 shutdown_connections 2023-07-06T01:19:51.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.638+0000 7fe02cb6f700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe028166a70 0x7fe028166ef0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.638+0000 7fe02cb6f700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe010004d90 0x7fe0280b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.638+0000 7fe02cb6f700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe00806ace0 0x7fe00806d190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.638+0000 7fe02cb6f700 1 --2- 172.21.15.22:0/631501495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe02814b830 0x7fe0280cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.638+0000 7fe02cb6f700 1 -- 172.21.15.22:0/631501495 >> 172.21.15.22:0/631501495 conn(0x7fe0280ccb00 msgr2=0x7fe028149fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:51.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.639+0000 7fe02cb6f700 1 -- 172.21.15.22:0/631501495 shutdown_connections 2023-07-06T01:19:51.644 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.639+0000 7fe02cb6f700 1 -- 172.21.15.22:0/631501495 wait complete. 2023-07-06T01:19:51.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:51.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.776+0000 7f5548b2c700 1 Processor -- start 2023-07-06T01:19:51.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.776+0000 7f5548b2c700 1 -- start start 2023-07-06T01:19:51.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5548b2c700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f55440b36f0 0x7f55440cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5548b2c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f55440cf5a0 0x7f55440d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5548b2c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55440b3b70 0x7f55440d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5548b2c700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554415cff0 con 0x7f55440cf5a0 2023-07-06T01:19:51.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5548b2c700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55440cebd0 con 0x7f55440b36f0 2023-07-06T01:19:51.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5548b2c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55440cdfc0 con 0x7f55440b3b70 2023-07-06T01:19:51.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5541d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f55440cf5a0 0x7f55440d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5541d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f55440cf5a0 0x7f55440d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46218/0 (socket says 172.21.15.22:46218) 2023-07-06T01:19:51.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5541d9b700 1 -- 172.21.15.22:0/675906559 learned_addr learned my addr 172.21.15.22:0/675906559 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:51.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f554259c700 1 --2- 172.21.15.22:0/675906559 >> v2:172.21.15.103:3300/0 conn(0x7f55440b36f0 0x7f55440cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5542d9d700 1 --2- 172.21.15.22:0/675906559 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55440b3b70 0x7f55440d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5541d9b700 1 -- 172.21.15.22:0/675906559 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55440b3b70 msgr2=0x7f55440d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:51.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5541d9b700 1 --2- 172.21.15.22:0/675906559 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55440b3b70 0x7f55440d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5541d9b700 1 -- 172.21.15.22:0/675906559 >> v2:172.21.15.103:3300/0 conn(0x7f55440b36f0 msgr2=0x7f55440cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:51.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5541d9b700 1 --2- 172.21.15.22:0/675906559 >> v2:172.21.15.103:3300/0 conn(0x7f55440b36f0 0x7f55440cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5541d9b700 1 -- 172.21.15.22:0/675906559 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55440ce2a0 con 0x7f55440cf5a0 2023-07-06T01:19:51.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f554259c700 1 --2- 172.21.15.22:0/675906559 >> v2:172.21.15.103:3300/0 conn(0x7f55440b36f0 0x7f55440cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:51.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.777+0000 7f5542d9d700 1 --2- 172.21.15.22:0/675906559 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55440b3b70 0x7f55440d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:51.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.778+0000 7f5541d9b700 1 --2- 172.21.15.22:0/675906559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55440cf5a0 0x7f55440d0a90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f554408b8d0 tx=0x7f5534009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5814b3da0e05ebf6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:51.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.778+0000 7f554159a700 1 -- 172.21.15.22:0/675906559 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553400b870 con 0x7f55440cf5a0 2023-07-06T01:19:51.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.778+0000 7f554159a700 1 -- 172.21.15.22:0/675906559 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553400b9d0 con 0x7f55440cf5a0 2023-07-06T01:19:51.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.778+0000 7f554159a700 1 -- 172.21.15.22:0/675906559 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553400bd60 con 0x7f55440cf5a0 2023-07-06T01:19:51.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.778+0000 7f5548b2c700 1 -- 172.21.15.22:0/675906559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55440cf5a0 msgr2=0x7f55440d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.778+0000 7f5548b2c700 1 --2- 172.21.15.22:0/675906559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55440cf5a0 0x7f55440d0a90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f554408b8d0 tx=0x7f5534009bd0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.778+0000 7f5548b2c700 1 -- 172.21.15.22:0/675906559 shutdown_connections 2023-07-06T01:19:51.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.778+0000 7f5548b2c700 1 --2- 172.21.15.22:0/675906559 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55440b3b70 0x7f55440d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.778+0000 7f5548b2c700 1 --2- 172.21.15.22:0/675906559 >> v2:172.21.15.103:3300/0 conn(0x7f55440b36f0 0x7f55440cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.778+0000 7f5548b2c700 1 --2- 172.21.15.22:0/675906559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55440cf5a0 0x7f55440d0a90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.778+0000 7f5548b2c700 1 -- 172.21.15.22:0/675906559 >> 172.21.15.22:0/675906559 conn(0x7f554409c2e0 msgr2=0x7f55440d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:51.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.778+0000 7f5548b2c700 1 -- 172.21.15.22:0/675906559 shutdown_connections 2023-07-06T01:19:51.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.778+0000 7f5548b2c700 1 -- 172.21.15.22:0/675906559 wait complete. 2023-07-06T01:19:51.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.780+0000 7f5548b2c700 1 Processor -- start 2023-07-06T01:19:51.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.780+0000 7f5548b2c700 1 -- start start 2023-07-06T01:19:51.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.780+0000 7f5548b2c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55440b36f0 0x7f55440cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f5548b2c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55440b3b70 0x7f55440cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f5548b2c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55440cf5a0 0x7f55440cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f5548b2c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554415e130 con 0x7f55440b36f0 2023-07-06T01:19:51.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f5548b2c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554415e410 con 0x7f55440b3b70 2023-07-06T01:19:51.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f5548b2c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554415e6f0 con 0x7f55440cf5a0 2023-07-06T01:19:51.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f554259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55440b36f0 0x7f55440cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f554259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55440b36f0 0x7f55440cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46222/0 (socket says 172.21.15.22:46222) 2023-07-06T01:19:51.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f554259c700 1 -- 172.21.15.22:0/3717032245 learned_addr learned my addr 172.21.15.22:0/3717032245 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:51.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f5541d9b700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55440b3b70 0x7f55440cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f5542d9d700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55440cf5a0 0x7f55440cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f554259c700 1 -- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55440cf5a0 msgr2=0x7f55440cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f554259c700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55440cf5a0 0x7f55440cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f554259c700 1 -- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55440b3b70 msgr2=0x7f55440cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f554259c700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55440b3b70 0x7f55440cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f554259c700 1 -- 172.21.15.22:0/3717032245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5534009550 con 0x7f55440b36f0 2023-07-06T01:19:51.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f5541d9b700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55440b3b70 0x7f55440cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:51.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f554259c700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55440b36f0 0x7f55440cc910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f553800c7f0 tx=0x7f553800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:51.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f5532ffd700 1 -- 172.21.15.22:0/3717032245 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538011800 con 0x7f55440b36f0 2023-07-06T01:19:51.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f5532ffd700 1 -- 172.21.15.22:0/3717032245 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5538011960 con 0x7f55440b36f0 2023-07-06T01:19:51.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f5548b2c700 1 -- 172.21.15.22:0/3717032245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554415e9d0 con 0x7f55440b36f0 2023-07-06T01:19:51.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f5532ffd700 1 -- 172.21.15.22:0/3717032245 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538011d10 con 0x7f55440b36f0 2023-07-06T01:19:51.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.781+0000 7f5548b2c700 1 -- 172.21.15.22:0/3717032245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554415c0c0 con 0x7f55440b36f0 2023-07-06T01:19:51.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.782+0000 7f5548b2c700 1 -- 172.21.15.22:0/3717032245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55440b4840 con 0x7f55440b36f0 2023-07-06T01:19:51.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.783+0000 7f5532ffd700 1 -- 172.21.15.22:0/3717032245 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5538018360 con 0x7f55440b36f0 2023-07-06T01:19:51.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.783+0000 7f5532ffd700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f552406ace0 0x7f552406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:51.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.783+0000 7f5532ffd700 1 -- 172.21.15.22:0/3717032245 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f55380894a0 con 0x7f55440b36f0 2023-07-06T01:19:51.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.783+0000 7f5541d9b700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f552406ace0 0x7f552406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:51.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.784+0000 7f5541d9b700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f552406ace0 0x7f552406d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f55440cf9a0 tx=0x7f5534008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:51.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.788+0000 7f5532ffd700 1 -- 172.21.15.22:0/3717032245 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f55380544c0 con 0x7f55440b36f0 2023-07-06T01:19:51.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.986+0000 7f5548b2c700 1 -- 172.21.15.22:0/3717032245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55440744b0 con 0x7f55440b36f0 2023-07-06T01:19:51.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.988+0000 7f5532ffd700 1 -- 172.21.15.22:0/3717032245 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7f5538057ae0 con 0x7f55440b36f0 2023-07-06T01:19:51.990 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:51.991 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:51.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.989+0000 7f5548b2c700 1 -- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f552406ace0 msgr2=0x7f552406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.989+0000 7f5548b2c700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f552406ace0 0x7f552406d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f55440cf9a0 tx=0x7f5534008030 comp rx=0 tx=0).stop 2023-07-06T01:19:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.989+0000 7f5548b2c700 1 -- 172.21.15.22:0/3717032245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55440b36f0 msgr2=0x7f55440cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.989+0000 7f5548b2c700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55440b36f0 0x7f55440cc910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f553800c7f0 tx=0x7f553800cbb0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.990+0000 7f5548b2c700 1 -- 172.21.15.22:0/3717032245 shutdown_connections 2023-07-06T01:19:51.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.990+0000 7f5548b2c700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55440cf5a0 0x7f55440cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.990+0000 7f5548b2c700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55440b36f0 0x7f55440cc910 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.990+0000 7f5548b2c700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f552406ace0 0x7f552406d190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.990+0000 7f5548b2c700 1 --2- 172.21.15.22:0/3717032245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55440b3b70 0x7f55440cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.990+0000 7f5548b2c700 1 -- 172.21.15.22:0/3717032245 >> 172.21.15.22:0/3717032245 conn(0x7f554409c2e0 msgr2=0x7f554415c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.990+0000 7f5548b2c700 1 -- 172.21.15.22:0/3717032245 shutdown_connections 2023-07-06T01:19:51.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:51.990+0000 7f5548b2c700 1 -- 172.21.15.22:0/3717032245 wait complete. 2023-07-06T01:19:52.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T01:19:22.873255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T01:19:24.019088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:19:52.006 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:19:52.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.132+0000 7fc661a34700 1 Processor -- start 2023-07-06T01:19:52.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.132+0000 7fc661a34700 1 -- start start 2023-07-06T01:19:52.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.133+0000 7fc661a34700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc65c0b36f0 0x7fc65c0cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.133+0000 7fc661a34700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc65c0cf5a0 0x7fc65c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.133+0000 7fc661a34700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65c0cebd0 con 0x7fc65c0b3b70 2023-07-06T01:19:52.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.133+0000 7fc661a34700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65c15cff0 con 0x7fc65c0cf5a0 2023-07-06T01:19:52.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.133+0000 7fc661a34700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0cdfc0 con 0x7fc65c0b36f0 2023-07-06T01:19:52.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.133+0000 7fc653fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc65c0cf5a0 0x7fc65c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.134+0000 7fc653fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc65c0cf5a0 0x7fc65c0d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46232/0 (socket says 172.21.15.22:46232) 2023-07-06T01:19:52.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.134+0000 7fc653fff700 1 -- 172.21.15.22:0/3757844721 learned_addr learned my addr 172.21.15.22:0/3757844721 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:52.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.134+0000 7fc65affd700 1 --2- 172.21.15.22:0/3757844721 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc65c0b36f0 0x7fc65c0cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.134+0000 7fc653fff700 1 -- 172.21.15.22:0/3757844721 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc65c0b36f0 msgr2=0x7fc65c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:52.138 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.134+0000 7fc653fff700 1 --2- 172.21.15.22:0/3757844721 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc65c0b36f0 0x7fc65c0cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.134+0000 7fc653fff700 1 -- 172.21.15.22:0/3757844721 >> v1:172.21.15.22:6789/0 conn(0x7fc65c0b3b70 legacy=0x7fc65c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.134+0000 7fc653fff700 1 -- 172.21.15.22:0/3757844721 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c0ce2a0 con 0x7fc65c0cf5a0 2023-07-06T01:19:52.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.134+0000 7fc653fff700 1 --2- 172.21.15.22:0/3757844721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc65c0cf5a0 0x7fc65c0d0a90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc6440098f0 tx=0x7fc644009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac668385caf2131e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:52.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.134+0000 7fc65a7fc700 1 -- 172.21.15.22:0/3757844721 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644011830 con 0x7fc65c0cf5a0 2023-07-06T01:19:52.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.134+0000 7fc65a7fc700 1 -- 172.21.15.22:0/3757844721 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644011990 con 0x7fc65c0cf5a0 2023-07-06T01:19:52.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.134+0000 7fc65a7fc700 1 -- 172.21.15.22:0/3757844721 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644011d20 con 0x7fc65c0cf5a0 2023-07-06T01:19:52.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.135+0000 7fc661a34700 1 -- 172.21.15.22:0/3757844721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc65c0cf5a0 msgr2=0x7fc65c0d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.135+0000 7fc661a34700 1 --2- 172.21.15.22:0/3757844721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc65c0cf5a0 0x7fc65c0d0a90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc6440098f0 tx=0x7fc644009c00 comp rx=0 tx=0).stop 2023-07-06T01:19:52.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.135+0000 7fc661a34700 1 -- 172.21.15.22:0/3757844721 shutdown_connections 2023-07-06T01:19:52.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.135+0000 7fc661a34700 1 --2- 172.21.15.22:0/3757844721 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc65c0b36f0 0x7fc65c0cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.135+0000 7fc661a34700 1 --2- 172.21.15.22:0/3757844721 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc65c0cf5a0 0x7fc65c0d0a90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.135+0000 7fc661a34700 1 -- 172.21.15.22:0/3757844721 >> 172.21.15.22:0/3757844721 conn(0x7fc65c09c2e0 msgr2=0x7fc65c0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:52.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.135+0000 7fc661a34700 1 -- 172.21.15.22:0/3757844721 shutdown_connections 2023-07-06T01:19:52.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.135+0000 7fc661a34700 1 -- 172.21.15.22:0/3757844721 wait complete. 2023-07-06T01:19:52.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.136+0000 7fc661a34700 1 Processor -- start 2023-07-06T01:19:52.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.136+0000 7fc661a34700 1 -- start start 2023-07-06T01:19:52.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.137+0000 7fc661a34700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc65c0b36f0 0x7fc65c15a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.137+0000 7fc661a34700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc65c0b3b70 0x7fc65c145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.137+0000 7fc661a34700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc65c0cf5a0 0x7fc65c143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.137+0000 7fc661a34700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c15a470 con 0x7fc65c0b3b70 2023-07-06T01:19:52.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.137+0000 7fc661a34700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c15a140 con 0x7fc65c0b36f0 2023-07-06T01:19:52.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.137+0000 7fc661a34700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0afb40 con 0x7fc65c0cf5a0 2023-07-06T01:19:52.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.137+0000 7fc653fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc65c0b3b70 0x7fc65c145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.137+0000 7fc653fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc65c0b3b70 0x7fc65c145b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46238/0 (socket says 172.21.15.22:46238) 2023-07-06T01:19:52.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.137+0000 7fc653fff700 1 -- 172.21.15.22:0/220859849 learned_addr learned my addr 172.21.15.22:0/220859849 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:52.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.138+0000 7fc65affd700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc65c0b36f0 0x7fc65c15a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.138+0000 7fc65b7fe700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc65c0cf5a0 0x7fc65c143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.138+0000 7fc653fff700 1 -- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc65c0cf5a0 msgr2=0x7fc65c143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.138+0000 7fc653fff700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc65c0cf5a0 0x7fc65c143f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.138+0000 7fc653fff700 1 -- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc65c0b36f0 msgr2=0x7fc65c15a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.138+0000 7fc653fff700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc65c0b36f0 0x7fc65c15a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.138+0000 7fc653fff700 1 -- 172.21.15.22:0/220859849 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc644009550 con 0x7fc65c0b3b70 2023-07-06T01:19:52.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.138+0000 7fc653fff700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc65c0b3b70 0x7fc65c145b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc644009c40 tx=0x7fc644004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:52.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.138+0000 7fc658ff9700 1 -- 172.21.15.22:0/220859849 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440036a0 con 0x7fc65c0b3b70 2023-07-06T01:19:52.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.138+0000 7fc658ff9700 1 -- 172.21.15.22:0/220859849 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64401e070 con 0x7fc65c0b3b70 2023-07-06T01:19:52.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.138+0000 7fc661a34700 1 -- 172.21.15.22:0/220859849 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65c0afe20 con 0x7fc65c0b3b70 2023-07-06T01:19:52.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.138+0000 7fc658ff9700 1 -- 172.21.15.22:0/220859849 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644003a60 con 0x7fc65c0b3b70 2023-07-06T01:19:52.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.138+0000 7fc661a34700 1 -- 172.21.15.22:0/220859849 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65c0b0100 con 0x7fc65c0b3b70 2023-07-06T01:19:52.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.139+0000 7fc661a34700 1 -- 172.21.15.22:0/220859849 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65c0b4840 con 0x7fc65c0b3b70 2023-07-06T01:19:52.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.139+0000 7fc658ff9700 1 -- 172.21.15.22:0/220859849 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc644003bc0 con 0x7fc65c0b3b70 2023-07-06T01:19:52.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.140+0000 7fc658ff9700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc63c06ace0 0x7fc63c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.140+0000 7fc658ff9700 1 -- 172.21.15.22:0/220859849 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc64408a3d0 con 0x7fc65c0b3b70 2023-07-06T01:19:52.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.140+0000 7fc65affd700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc63c06ace0 0x7fc63c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.140+0000 7fc65affd700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc63c06ace0 0x7fc63c06d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009b70 tx=0x7fc64c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:52.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.144+0000 7fc658ff9700 1 -- 172.21.15.22:0/220859849 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc644055510 con 0x7fc65c0b3b70 2023-07-06T01:19:52.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.344+0000 7fc661a34700 1 -- 172.21.15.22:0/220859849 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc65c159f80 con 0x7fc65c0b3b70 2023-07-06T01:19:52.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.345+0000 7fc658ff9700 1 -- 172.21.15.22:0/220859849 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc644058b30 con 0x7fc65c0b3b70 2023-07-06T01:19:52.348 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:19:52.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.347+0000 7fc661a34700 1 -- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc63c06ace0 msgr2=0x7fc63c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.347+0000 7fc661a34700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc63c06ace0 0x7fc63c06d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009b70 tx=0x7fc64c008040 comp rx=0 tx=0).stop 2023-07-06T01:19:52.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.347+0000 7fc661a34700 1 -- 172.21.15.22:0/220859849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc65c0b3b70 msgr2=0x7fc65c145b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.347+0000 7fc661a34700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc65c0b3b70 0x7fc65c145b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc644009c40 tx=0x7fc644004750 comp rx=0 tx=0).stop 2023-07-06T01:19:52.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.347+0000 7fc661a34700 1 -- 172.21.15.22:0/220859849 shutdown_connections 2023-07-06T01:19:52.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.347+0000 7fc661a34700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc65c0cf5a0 0x7fc65c143f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.347+0000 7fc661a34700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc63c06ace0 0x7fc63c06d190 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.347+0000 7fc661a34700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc65c0b36f0 0x7fc65c15a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.347+0000 7fc661a34700 1 --2- 172.21.15.22:0/220859849 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc65c0b3b70 0x7fc65c145b80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.347+0000 7fc661a34700 1 -- 172.21.15.22:0/220859849 >> 172.21.15.22:0/220859849 conn(0x7fc65c09c2e0 msgr2=0x7fc65c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:52.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.347+0000 7fc661a34700 1 -- 172.21.15.22:0/220859849 shutdown_connections 2023-07-06T01:19:52.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.347+0000 7fc661a34700 1 -- 172.21.15.22:0/220859849 wait complete. 2023-07-06T01:19:52.362 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:19:52.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.487+0000 7f1d502d3700 1 Processor -- start 2023-07-06T01:19:52.490 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.487+0000 7f1d502d3700 1 -- start start 2023-07-06T01:19:52.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.488+0000 7f1d502d3700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d481597a0 0x7f1d4806cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.488+0000 7f1d502d3700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f1d480c9a50 0x7f1d480c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.488+0000 7f1d502d3700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d480ccf30 con 0x7f1d480cc750 2023-07-06T01:19:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.488+0000 7f1d502d3700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d480ca410 con 0x7f1d480c9a50 2023-07-06T01:19:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.488+0000 7f1d502d3700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480cd210 con 0x7f1d481597a0 2023-07-06T01:19:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.488+0000 7f1d4d86e700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1d480cc750 0x7f1d480cdfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36988/0 (socket says 172.21.15.22:36988) 2023-07-06T01:19:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.488+0000 7f1d4d86e700 1 -- 172.21.15.22:0/2847877612 learned_addr learned my addr 172.21.15.22:0/2847877612 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:52.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.488+0000 7f1d4e870700 1 --2- 172.21.15.22:0/2847877612 >> v2:172.21.15.103:3300/0 conn(0x7f1d480c9a50 0x7f1d480c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.488+0000 7f1d4e06f700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d481597a0 0x7f1d4806cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.488+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733357876 0 0) 0x7f1d480ccf30 con 0x7f1d480cc750 2023-07-06T01:19:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.488+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d340035b0 con 0x7f1d480cc750 2023-07-06T01:19:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 286448271 0 0) 0x7f1d340035b0 con 0x7f1d480cc750 2023-07-06T01:19:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d481597a0 msgr2=0x7f1d4806cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d481597a0 0x7f1d4806cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.494 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 >> v2:172.21.15.103:3300/0 conn(0x7f1d480c9a50 msgr2=0x7f1d480c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 --2- 172.21.15.22:0/2847877612 >> v2:172.21.15.103:3300/0 conn(0x7f1d480c9a50 0x7f1d480c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d480cd4f0 con 0x7f1d480cc750 2023-07-06T01:19:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f1d44001670 con 0x7f1d480cc750 2023-07-06T01:19:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4e870700 1 --2- 172.21.15.22:0/2847877612 >> v2:172.21.15.103:3300/0 conn(0x7f1d480c9a50 0x7f1d480c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 >> v1:172.21.15.22:6789/0 conn(0x7f1d480cc750 legacy=0x7f1d480cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d34003a00 0x7f1d34003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d34004850 0x7f1d34004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d481597a0 0x7f1d34005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480ca410 con 0x7f1d34003a00 2023-07-06T01:19:52.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480cd210 con 0x7f1d34004850 2023-07-06T01:19:52.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3400ba90 con 0x7f1d481597a0 2023-07-06T01:19:52.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4e06f700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d34003a00 0x7f1d34003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d86e700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d34004850 0x7f1d34004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4e870700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d481597a0 0x7f1d34005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4e06f700 1 -- 172.21.15.22:0/2847877612 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d481597a0 msgr2=0x7f1d34005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:52.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4e06f700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d481597a0 0x7f1d34005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4e06f700 1 -- 172.21.15.22:0/2847877612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d34004850 msgr2=0x7f1d34004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:52.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4e06f700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d34004850 0x7f1d34004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4e06f700 1 -- 172.21.15.22:0/2847877612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d340035b0 con 0x7f1d34003a00 2023-07-06T01:19:52.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4e870700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d481597a0 0x7f1d34005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:52.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d86e700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d34004850 0x7f1d34004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:52.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4e06f700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d34003a00 0x7f1d34003e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009b60 tx=0x7f1d34005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c9c10c741ce4cb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:52.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800e6c0 con 0x7f1d34003a00 2023-07-06T01:19:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3800e820 con 0x7f1d34003a00 2023-07-06T01:19:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.489+0000 7f1d4d06d700 1 -- 172.21.15.22:0/2847877612 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800ebb0 con 0x7f1d34003a00 2023-07-06T01:19:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.490+0000 7f1d502d3700 1 -- 172.21.15.22:0/2847877612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d34003a00 msgr2=0x7f1d34003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.490+0000 7f1d502d3700 1 --2- 172.21.15.22:0/2847877612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d34003a00 0x7f1d34003e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009b60 tx=0x7f1d34005880 comp rx=0 tx=0).stop 2023-07-06T01:19:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.490+0000 7f1d4e870700 1 -- 172.21.15.22:0/2847877612 reap_dead start 2023-07-06T01:19:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.490+0000 7f1d502d3700 1 -- 172.21.15.22:0/2847877612 shutdown_connections 2023-07-06T01:19:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.490+0000 7f1d502d3700 1 -- 172.21.15.22:0/2847877612 >> 172.21.15.22:0/2847877612 conn(0x7f1d480b0710 msgr2=0x7f1d480b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:52.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.490+0000 7f1d502d3700 1 -- 172.21.15.22:0/2847877612 shutdown_connections 2023-07-06T01:19:52.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.490+0000 7f1d502d3700 1 -- 172.21.15.22:0/2847877612 wait complete. 2023-07-06T01:19:52.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.491+0000 7f1d502d3700 1 Processor -- start 2023-07-06T01:19:52.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.491+0000 7f1d502d3700 1 -- start start 2023-07-06T01:19:52.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d502d3700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d34003a00 0x7f1d481eeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d502d3700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d48159e70 0x7f1d481eb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d502d3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d4806ba40 0x7f1d4806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d502d3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4806c400 con 0x7f1d4806ba40 2023-07-06T01:19:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d502d3700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d481ef0a0 con 0x7f1d48159e70 2023-07-06T01:19:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d502d3700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480b2d30 con 0x7f1d34003a00 2023-07-06T01:19:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d4e870700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d4806ba40 0x7f1d4806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d4e870700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d4806ba40 0x7f1d4806bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46260/0 (socket says 172.21.15.22:46260) 2023-07-06T01:19:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d4e870700 1 -- 172.21.15.22:0/2450780022 learned_addr learned my addr 172.21.15.22:0/2450780022 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d4d86e700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d48159e70 0x7f1d481eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d4e06f700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d34003a00 0x7f1d481eeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d4e870700 1 -- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d34003a00 msgr2=0x7f1d481eeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d4e870700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d34003a00 0x7f1d481eeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d4e870700 1 -- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d48159e70 msgr2=0x7f1d481eb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d4e870700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d48159e70 0x7f1d481eb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.492+0000 7f1d4e870700 1 -- 172.21.15.22:0/2450780022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d480b3010 con 0x7f1d4806ba40 2023-07-06T01:19:52.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.493+0000 7f1d4e870700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d4806ba40 0x7f1d4806bec0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1d48068310 tx=0x7f1d40011790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:52.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.493+0000 7f1d3effd700 1 -- 172.21.15.22:0/2450780022 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4000e070 con 0x7f1d4806ba40 2023-07-06T01:19:52.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.493+0000 7f1d3effd700 1 -- 172.21.15.22:0/2450780022 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d40003a20 con 0x7f1d4806ba40 2023-07-06T01:19:52.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.493+0000 7f1d502d3700 1 -- 172.21.15.22:0/2450780022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d480b32f0 con 0x7f1d4806ba40 2023-07-06T01:19:52.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.493+0000 7f1d3effd700 1 -- 172.21.15.22:0/2450780022 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d40003bd0 con 0x7f1d4806ba40 2023-07-06T01:19:52.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.493+0000 7f1d502d3700 1 -- 172.21.15.22:0/2450780022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d480b38b0 con 0x7f1d4806ba40 2023-07-06T01:19:52.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.494+0000 7f1d502d3700 1 -- 172.21.15.22:0/2450780022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d480744b0 con 0x7f1d4806ba40 2023-07-06T01:19:52.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.494+0000 7f1d3effd700 1 -- 172.21.15.22:0/2450780022 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d40003d30 con 0x7f1d4806ba40 2023-07-06T01:19:52.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.495+0000 7f1d3effd700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d2c06ace0 0x7f1d2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.495+0000 7f1d3effd700 1 -- 172.21.15.22:0/2450780022 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1d40089a10 con 0x7f1d4806ba40 2023-07-06T01:19:52.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.495+0000 7f1d4e06f700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d2c06ace0 0x7f1d2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.495+0000 7f1d4e06f700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d2c06ace0 0x7f1d2c06d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1d380090b0 tx=0x7f1d38015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:52.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.499+0000 7f1d3effd700 1 -- 172.21.15.22:0/2450780022 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1d40054ac0 con 0x7f1d4806ba40 2023-07-06T01:19:52.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.700+0000 7f1d502d3700 1 -- 172.21.15.22:0/2450780022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d480aefc0 con 0x7f1d4806ba40 2023-07-06T01:19:52.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.700+0000 7f1d3effd700 1 -- 172.21.15.22:0/2450780022 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d400580e0 con 0x7f1d4806ba40 2023-07-06T01:19:52.703 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:19:52.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.702+0000 7f1d502d3700 1 -- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d2c06ace0 msgr2=0x7f1d2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.702+0000 7f1d502d3700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d2c06ace0 0x7f1d2c06d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1d380090b0 tx=0x7f1d38015000 comp rx=0 tx=0).stop 2023-07-06T01:19:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.702+0000 7f1d502d3700 1 -- 172.21.15.22:0/2450780022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d4806ba40 msgr2=0x7f1d4806bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.702+0000 7f1d502d3700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d4806ba40 0x7f1d4806bec0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1d48068310 tx=0x7f1d40011790 comp rx=0 tx=0).stop 2023-07-06T01:19:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.702+0000 7f1d502d3700 1 -- 172.21.15.22:0/2450780022 shutdown_connections 2023-07-06T01:19:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.702+0000 7f1d502d3700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d4806ba40 0x7f1d4806bec0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.702+0000 7f1d502d3700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d34003a00 0x7f1d481eeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.703+0000 7f1d502d3700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d2c06ace0 0x7f1d2c06d190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.703+0000 7f1d502d3700 1 --2- 172.21.15.22:0/2450780022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d48159e70 0x7f1d481eb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.703+0000 7f1d502d3700 1 -- 172.21.15.22:0/2450780022 >> 172.21.15.22:0/2450780022 conn(0x7f1d480b0710 msgr2=0x7f1d480af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.703+0000 7f1d502d3700 1 -- 172.21.15.22:0/2450780022 shutdown_connections 2023-07-06T01:19:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.703+0000 7f1d502d3700 1 -- 172.21.15.22:0/2450780022 wait complete. 2023-07-06T01:19:52.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T01:19:52.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.842+0000 7f7f369ef700 1 Processor -- start 2023-07-06T01:19:52.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.842+0000 7f7f369ef700 1 -- start start 2023-07-06T01:19:52.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.843+0000 7f7f369ef700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7f300cc320 0x7f7f3006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.843+0000 7f7f369ef700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f300af670 con 0x7f7f300af2a0 2023-07-06T01:19:52.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.843+0000 7f7f369ef700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3006d030 con 0x7f7f3006a710 2023-07-06T01:19:52.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.843+0000 7f7f369ef700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f300cc780 con 0x7f7f300cc320 2023-07-06T01:19:52.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.843+0000 7f7f2ffff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f7f300af2a0 0x7f7f3006a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37008/0 (socket says 172.21.15.22:37008) 2023-07-06T01:19:52.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.843+0000 7f7f2ffff700 1 -- 172.21.15.22:0/815865480 learned_addr learned my addr 172.21.15.22:0/815865480 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:52.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.843+0000 7f7f34f8c700 1 --2- 172.21.15.22:0/815865480 >> v2:172.21.15.103:3300/0 conn(0x7f7f300cc320 0x7f7f3006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.843+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909602169 0 0) 0x7f7f300af670 con 0x7f7f300af2a0 2023-07-06T01:19:52.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f1c0035b0 con 0x7f7f300af2a0 2023-07-06T01:19:52.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3518255521 0 0) 0x7f7f1c0035b0 con 0x7f7f300af2a0 2023-07-06T01:19:52.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 >> v2:172.21.15.103:3300/0 conn(0x7f7f300cc320 msgr2=0x7f7f3006caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:52.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 --2- 172.21.15.22:0/815865480 >> v2:172.21.15.103:3300/0 conn(0x7f7f300cc320 0x7f7f3006caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 >> v1:172.21.15.103:6789/0 conn(0x7f7f3006a710 legacy=0x7f7f300cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f300cd560 con 0x7f7f300af2a0 2023-07-06T01:19:52.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f7f18001670 con 0x7f7f300af2a0 2023-07-06T01:19:52.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 >> v1:172.21.15.22:6789/0 conn(0x7f7f300af2a0 legacy=0x7f7f3006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 --2- 172.21.15.22:0/815865480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f1c003a00 0x7f7f1c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 --2- 172.21.15.22:0/815865480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c004850 0x7f7f1c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 --2- 172.21.15.22:0/815865480 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f1c005200 0x7f7f1c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f300cc780 con 0x7f7f1c004850 2023-07-06T01:19:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c00c080 con 0x7f7f1c003a00 2023-07-06T01:19:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c00c240 con 0x7f7f1c005200 2023-07-06T01:19:52.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2f7fe700 1 --2- 172.21.15.22:0/815865480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c004850 0x7f7f1c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2ffff700 1 --2- 172.21.15.22:0/815865480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f1c003a00 0x7f7f1c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f34f8c700 1 --2- 172.21.15.22:0/815865480 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f1c005200 0x7f7f1c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2f7fe700 1 -- 172.21.15.22:0/815865480 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f1c005200 msgr2=0x7f7f1c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:52.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2f7fe700 1 --2- 172.21.15.22:0/815865480 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f1c005200 0x7f7f1c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2f7fe700 1 -- 172.21.15.22:0/815865480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f1c003a00 msgr2=0x7f7f1c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:52.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2f7fe700 1 --2- 172.21.15.22:0/815865480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f1c003a00 0x7f7f1c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f34f8c700 1 -- 172.21.15.22:0/815865480 reap_dead start 2023-07-06T01:19:52.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2f7fe700 1 -- 172.21.15.22:0/815865480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c0035b0 con 0x7f7f1c004850 2023-07-06T01:19:52.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.844+0000 7f7f2ffff700 1 --2- 172.21.15.22:0/815865480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f1c003a00 0x7f7f1c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:19:52.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.845+0000 7f7f2f7fe700 1 --2- 172.21.15.22:0/815865480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c004850 0x7f7f1c004cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f300ca2a0 tx=0x7f7f1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d007c4ee1e0eb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:52.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.845+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f200052d0 con 0x7f7f1c004850 2023-07-06T01:19:52.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.845+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f20005450 con 0x7f7f1c004850 2023-07-06T01:19:52.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.845+0000 7f7f2effd700 1 -- 172.21.15.22:0/815865480 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f200038c0 con 0x7f7f1c004850 2023-07-06T01:19:52.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.845+0000 7f7f369ef700 1 -- 172.21.15.22:0/815865480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c004850 msgr2=0x7f7f1c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.845+0000 7f7f369ef700 1 --2- 172.21.15.22:0/815865480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c004850 0x7f7f1c004cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f300ca2a0 tx=0x7f7f1c005880 comp rx=0 tx=0).stop 2023-07-06T01:19:52.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.845+0000 7f7f369ef700 1 -- 172.21.15.22:0/815865480 shutdown_connections 2023-07-06T01:19:52.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.845+0000 7f7f369ef700 1 --2- 172.21.15.22:0/815865480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f1c004850 0x7f7f1c004cc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.845+0000 7f7f369ef700 1 -- 172.21.15.22:0/815865480 >> 172.21.15.22:0/815865480 conn(0x7f7f300b1890 msgr2=0x7f7f300b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:52.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.845+0000 7f7f369ef700 1 -- 172.21.15.22:0/815865480 shutdown_connections 2023-07-06T01:19:52.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.845+0000 7f7f369ef700 1 -- 172.21.15.22:0/815865480 wait complete. 2023-07-06T01:19:52.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.847+0000 7f7f369ef700 1 Processor -- start 2023-07-06T01:19:52.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.847+0000 7f7f369ef700 1 -- start start 2023-07-06T01:19:52.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.848+0000 7f7f369ef700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f1c004850 0x7f7f301f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.848+0000 7f7f369ef700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f301475c0 0x7f7f30148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.848+0000 7f7f369ef700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f301491e0 0x7f7f30149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.848+0000 7f7f369ef700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f301489a0 con 0x7f7f301491e0 2023-07-06T01:19:52.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.848+0000 7f7f369ef700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f30147d90 con 0x7f7f1c004850 2023-07-06T01:19:52.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.848+0000 7f7f369ef700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f30148070 con 0x7f7f301475c0 2023-07-06T01:19:52.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.848+0000 7f7f34f8c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f301491e0 0x7f7f30149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.848+0000 7f7f34f8c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f301491e0 0x7f7f30149650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46280/0 (socket says 172.21.15.22:46280) 2023-07-06T01:19:52.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.848+0000 7f7f34f8c700 1 -- 172.21.15.22:0/41046560 learned_addr learned my addr 172.21.15.22:0/41046560 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:52.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.848+0000 7f7f2ffff700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f1c004850 0x7f7f301f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.848+0000 7f7f2f7fe700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f301475c0 0x7f7f30148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.849+0000 7f7f34f8c700 1 -- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f301475c0 msgr2=0x7f7f30148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.849+0000 7f7f34f8c700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f301475c0 0x7f7f30148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.849+0000 7f7f34f8c700 1 -- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f1c004850 msgr2=0x7f7f301f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:52.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.849+0000 7f7f34f8c700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f1c004850 0x7f7f301f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.849+0000 7f7f34f8c700 1 -- 172.21.15.22:0/41046560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f30148350 con 0x7f7f301491e0 2023-07-06T01:19:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.849+0000 7f7f2f7fe700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f301475c0 0x7f7f30148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.849+0000 7f7f34f8c700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f301491e0 0x7f7f30149650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7f300cb2f0 tx=0x7f7f28011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.849+0000 7f7f2cff9700 1 -- 172.21.15.22:0/41046560 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800e040 con 0x7f7f301491e0 2023-07-06T01:19:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.849+0000 7f7f2cff9700 1 -- 172.21.15.22:0/41046560 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2800f040 con 0x7f7f301491e0 2023-07-06T01:19:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.849+0000 7f7f369ef700 1 -- 172.21.15.22:0/41046560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f30148630 con 0x7f7f301491e0 2023-07-06T01:19:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.849+0000 7f7f2cff9700 1 -- 172.21.15.22:0/41046560 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800fd00 con 0x7f7f301491e0 2023-07-06T01:19:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.849+0000 7f7f369ef700 1 -- 172.21.15.22:0/41046560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3015cec0 con 0x7f7f301491e0 2023-07-06T01:19:52.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.850+0000 7f7f369ef700 1 -- 172.21.15.22:0/41046560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f300b5cc0 con 0x7f7f301491e0 2023-07-06T01:19:52.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.850+0000 7f7f2cff9700 1 -- 172.21.15.22:0/41046560 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f28022460 con 0x7f7f301491e0 2023-07-06T01:19:52.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.851+0000 7f7f2cff9700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f1006ace0 0x7f7f1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:52.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.851+0000 7f7f2cff9700 1 -- 172.21.15.22:0/41046560 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7f28089a40 con 0x7f7f301491e0 2023-07-06T01:19:52.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.851+0000 7f7f2ffff700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f1006ace0 0x7f7f1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:52.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.851+0000 7f7f2ffff700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f1006ace0 0x7f7f1006d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7f3006ed40 tx=0x7f7f180011c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:52.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:52.855+0000 7f7f2cff9700 1 -- 172.21.15.22:0/41046560 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7f2808be60 con 0x7f7f301491e0 2023-07-06T01:19:53.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.051+0000 7f7f369ef700 1 -- 172.21.15.22:0/41046560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f3015ccc0 con 0x7f7f301491e0 2023-07-06T01:19:53.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.051+0000 7f7f2cff9700 1 -- 172.21.15.22:0/41046560 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f28058580 con 0x7f7f301491e0 2023-07-06T01:19:53.054 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-07-06T01:19:53.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.053+0000 7f7f369ef700 1 -- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f1006ace0 msgr2=0x7f7f1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.053+0000 7f7f369ef700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f1006ace0 0x7f7f1006d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7f3006ed40 tx=0x7f7f180011c0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.053+0000 7f7f369ef700 1 -- 172.21.15.22:0/41046560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f301491e0 msgr2=0x7f7f30149650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.053+0000 7f7f369ef700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f301491e0 0x7f7f30149650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7f300cb2f0 tx=0x7f7f28011ed0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.053+0000 7f7f369ef700 1 -- 172.21.15.22:0/41046560 shutdown_connections 2023-07-06T01:19:53.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.053+0000 7f7f369ef700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f301491e0 0x7f7f30149650 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.053+0000 7f7f369ef700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f1006ace0 0x7f7f1006d190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.053+0000 7f7f369ef700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f1c004850 0x7f7f301f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.053+0000 7f7f369ef700 1 --2- 172.21.15.22:0/41046560 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7f301475c0 0x7f7f30148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.053+0000 7f7f369ef700 1 -- 172.21.15.22:0/41046560 >> 172.21.15.22:0/41046560 conn(0x7f7f300b1890 msgr2=0x7f7f300ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:53.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.053+0000 7f7f369ef700 1 -- 172.21.15.22:0/41046560 shutdown_connections 2023-07-06T01:19:53.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.053+0000 7f7f369ef700 1 -- 172.21.15.22:0/41046560 wait complete. 2023-07-06T01:19:53.068 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T01:19:53.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.192+0000 7f82f3fc6700 1 Processor -- start 2023-07-06T01:19:53.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.193+0000 7f82f3fc6700 1 -- start start 2023-07-06T01:19:53.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.193+0000 7f82f3fc6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f82ec06b320 0x7f82ec06c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.193+0000 7f82f3fc6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f82ec06c730 0x7f82ec0c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.193+0000 7f82f3fc6700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82ec071f30 con 0x7f82ec15a610 2023-07-06T01:19:53.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.193+0000 7f82f3fc6700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ec15a9e0 con 0x7f82ec06b320 2023-07-06T01:19:53.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.193+0000 7f82f3fc6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec06cbb0 con 0x7f82ec06c730 2023-07-06T01:19:53.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.193+0000 7f82f1561700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f82ec06b320 0x7f82ec06c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.194+0000 7f82f1561700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f82ec06b320 0x7f82ec06c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46286/0 (socket says 172.21.15.22:46286) 2023-07-06T01:19:53.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.194+0000 7f82f1561700 1 -- 172.21.15.22:0/1625125604 learned_addr learned my addr 172.21.15.22:0/1625125604 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:53.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.194+0000 7f82f2563700 1 --2- 172.21.15.22:0/1625125604 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f82ec06c730 0x7f82ec0c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.194+0000 7f82f1561700 1 -- 172.21.15.22:0/1625125604 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f82ec06c730 msgr2=0x7f82ec0c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:53.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.194+0000 7f82f1561700 1 --2- 172.21.15.22:0/1625125604 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f82ec06c730 0x7f82ec0c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.194+0000 7f82f1561700 1 -- 172.21.15.22:0/1625125604 >> v1:172.21.15.103:6789/0 conn(0x7f82ec15a610 legacy=0x7f82ec15eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.194+0000 7f82f1561700 1 -- 172.21.15.22:0/1625125604 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec06b770 con 0x7f82ec06b320 2023-07-06T01:19:53.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.194+0000 7f82f2563700 1 --2- 172.21.15.22:0/1625125604 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f82ec06c730 0x7f82ec0c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:53.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.194+0000 7f82f1561700 1 --2- 172.21.15.22:0/1625125604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82ec06b320 0x7f82ec06c1f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f82e80098f0 tx=0x7f82e8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7480e402a1f0eff8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:53.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.194+0000 7f82f0d60700 1 -- 172.21.15.22:0/1625125604 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e800b8a0 con 0x7f82ec06b320 2023-07-06T01:19:53.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.194+0000 7f82f0d60700 1 -- 172.21.15.22:0/1625125604 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e800ba00 con 0x7f82ec06b320 2023-07-06T01:19:53.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.194+0000 7f82f0d60700 1 -- 172.21.15.22:0/1625125604 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e800bd90 con 0x7f82ec06b320 2023-07-06T01:19:53.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.195+0000 7f82f3fc6700 1 -- 172.21.15.22:0/1625125604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82ec06b320 msgr2=0x7f82ec06c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.195+0000 7f82f3fc6700 1 --2- 172.21.15.22:0/1625125604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82ec06b320 0x7f82ec06c1f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f82e80098f0 tx=0x7f82e8009c00 comp rx=0 tx=0).stop 2023-07-06T01:19:53.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.195+0000 7f82f3fc6700 1 -- 172.21.15.22:0/1625125604 shutdown_connections 2023-07-06T01:19:53.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.195+0000 7f82f3fc6700 1 --2- 172.21.15.22:0/1625125604 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f82ec06c730 0x7f82ec0c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.195+0000 7f82f3fc6700 1 --2- 172.21.15.22:0/1625125604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82ec06b320 0x7f82ec06c1f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.195+0000 7f82f3fc6700 1 -- 172.21.15.22:0/1625125604 >> 172.21.15.22:0/1625125604 conn(0x7f82ec0ccb00 msgr2=0x7f82ec0cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:53.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.195+0000 7f82f3fc6700 1 -- 172.21.15.22:0/1625125604 shutdown_connections 2023-07-06T01:19:53.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.195+0000 7f82f3fc6700 1 -- 172.21.15.22:0/1625125604 wait complete. 2023-07-06T01:19:53.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.196+0000 7f82f3fc6700 1 Processor -- start 2023-07-06T01:19:53.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.196+0000 7f82f3fc6700 1 -- start start 2023-07-06T01:19:53.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f3fc6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82ec06b320 0x7f82ec1e5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f3fc6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f82ec06c730 0x7f82ec1e4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f3fc6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82ec15a610 0x7f82ec0b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f3fc6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec1e64a0 con 0x7f82ec06b320 2023-07-06T01:19:53.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f3fc6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec1e4fd0 con 0x7f82ec15a610 2023-07-06T01:19:53.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f3fc6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec1e5820 con 0x7f82ec06c730 2023-07-06T01:19:53.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f1d62700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82ec06b320 0x7f82ec1e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f1d62700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82ec06b320 0x7f82ec1e5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46290/0 (socket says 172.21.15.22:46290) 2023-07-06T01:19:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f1d62700 1 -- 172.21.15.22:0/228937519 learned_addr learned my addr 172.21.15.22:0/228937519 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f2563700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82ec15a610 0x7f82ec0b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f1561700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f82ec06c730 0x7f82ec1e4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f1d62700 1 -- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f82ec06c730 msgr2=0x7f82ec1e4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f1d62700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f82ec06c730 0x7f82ec1e4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f1d62700 1 -- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82ec15a610 msgr2=0x7f82ec0b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f1d62700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82ec15a610 0x7f82ec0b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f1d62700 1 -- 172.21.15.22:0/228937519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e8009550 con 0x7f82ec06b320 2023-07-06T01:19:53.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f1561700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f82ec06c730 0x7f82ec1e4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:53.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82f1d62700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82ec06b320 0x7f82ec1e5180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00c450 tx=0x7f82dc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:53.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82e27fc700 1 -- 172.21.15.22:0/228937519 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0052d0 con 0x7f82ec06b320 2023-07-06T01:19:53.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.197+0000 7f82e27fc700 1 -- 172.21.15.22:0/228937519 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc005430 con 0x7f82ec06b320 2023-07-06T01:19:53.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.198+0000 7f82e27fc700 1 -- 172.21.15.22:0/228937519 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0038c0 con 0x7f82ec06b320 2023-07-06T01:19:53.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.198+0000 7f82f3fc6700 1 -- 172.21.15.22:0/228937519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ec1e5b00 con 0x7f82ec06b320 2023-07-06T01:19:53.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.198+0000 7f82f3fc6700 1 -- 172.21.15.22:0/228937519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ec1e60c0 con 0x7f82ec06b320 2023-07-06T01:19:53.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.198+0000 7f82f3fc6700 1 -- 172.21.15.22:0/228937519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ec0744b0 con 0x7f82ec06b320 2023-07-06T01:19:53.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.199+0000 7f82e27fc700 1 -- 172.21.15.22:0/228937519 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f82dc003a20 con 0x7f82ec06b320 2023-07-06T01:19:53.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.199+0000 7f82e27fc700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f82d006acb0 0x7f82d006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.199+0000 7f82e27fc700 1 -- 172.21.15.22:0/228937519 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82dc0887c0 con 0x7f82ec06b320 2023-07-06T01:19:53.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.200+0000 7f82f1561700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f82d006acb0 0x7f82d006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.200+0000 7f82f1561700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f82d006acb0 0x7f82d006d160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009c40 tx=0x7f82e8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:53.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.204+0000 7f82e27fc700 1 -- 172.21.15.22:0/228937519 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f82dc053900 con 0x7f82ec06b320 2023-07-06T01:19:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.401+0000 7f82f3fc6700 1 -- 172.21.15.22:0/228937519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82ec0cfbd0 con 0x7f82ec06b320 2023-07-06T01:19:53.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.401+0000 7f82e27fc700 1 -- 172.21.15.22:0/228937519 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f82dc056f20 con 0x7f82ec06b320 2023-07-06T01:19:53.404 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-07-06T01:19:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.403+0000 7f82f3fc6700 1 -- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f82d006acb0 msgr2=0x7f82d006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.403+0000 7f82f3fc6700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f82d006acb0 0x7f82d006d160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009c40 tx=0x7f82e8008030 comp rx=0 tx=0).stop 2023-07-06T01:19:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.403+0000 7f82f3fc6700 1 -- 172.21.15.22:0/228937519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82ec06b320 msgr2=0x7f82ec1e5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.403+0000 7f82f3fc6700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82ec06b320 0x7f82ec1e5180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00c450 tx=0x7f82dc00c760 comp rx=0 tx=0).stop 2023-07-06T01:19:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.403+0000 7f82f3fc6700 1 -- 172.21.15.22:0/228937519 shutdown_connections 2023-07-06T01:19:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.403+0000 7f82f3fc6700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f82d006acb0 0x7f82d006d160 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.403+0000 7f82f3fc6700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82ec15a610 0x7f82ec0b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.403+0000 7f82f3fc6700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82ec06b320 0x7f82ec1e5180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.403+0000 7f82f3fc6700 1 --2- 172.21.15.22:0/228937519 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f82ec06c730 0x7f82ec1e4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.403+0000 7f82f3fc6700 1 -- 172.21.15.22:0/228937519 >> 172.21.15.22:0/228937519 conn(0x7f82ec0ccb00 msgr2=0x7f82ec06abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.403+0000 7f82f3fc6700 1 -- 172.21.15.22:0/228937519 shutdown_connections 2023-07-06T01:19:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.403+0000 7f82f3fc6700 1 -- 172.21.15.22:0/228937519 wait complete. 2023-07-06T01:19:53.418 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:53.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.540+0000 7fdf0297e700 1 Processor -- start 2023-07-06T01:19:53.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.540+0000 7fdf0297e700 1 -- start start 2023-07-06T01:19:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.540+0000 7fdf0297e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdefc15a710 0x7fdefc0ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdf0297e700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fdefc0cef70 0x7fdefc0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdf0297e700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc0af270 con 0x7fdefc0aecb0 2023-07-06T01:19:53.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdf0297e700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc0af550 con 0x7fdefc0cef70 2023-07-06T01:19:53.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdf0297e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0af830 con 0x7fdefc15a710 2023-07-06T01:19:53.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdf00f1b700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fdefc0cef70 0x7fdefc0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdefbfff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdefc15a710 0x7fdefc0ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdefb7fe700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fdefc0aecb0 0x7fdefc0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:39108/0 (socket says 172.21.15.22:39108) 2023-07-06T01:19:53.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdefb7fe700 1 -- 172.21.15.22:0/4132624454 learned_addr learned my addr 172.21.15.22:0/4132624454 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:53.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdefaffd700 1 -- 172.21.15.22:0/4132624454 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665940278 0 0) 0x7fdefc0af270 con 0x7fdefc0aecb0 2023-07-06T01:19:53.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdefaffd700 1 -- 172.21.15.22:0/4132624454 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdee40035b0 con 0x7fdefc0aecb0 2023-07-06T01:19:53.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdf00f1b700 1 -- 172.21.15.22:0/4132624454 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdefc15a710 msgr2=0x7fdefc0ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:53.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdf00f1b700 1 --2- 172.21.15.22:0/4132624454 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdefc15a710 0x7fdefc0ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdf00f1b700 1 -- 172.21.15.22:0/4132624454 >> v1:172.21.15.103:6789/0 conn(0x7fdefc0aecb0 legacy=0x7fdefc0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdf00f1b700 1 -- 172.21.15.22:0/4132624454 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc0cb2e0 con 0x7fdefc0cef70 2023-07-06T01:19:53.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.541+0000 7fdf00f1b700 1 --2- 172.21.15.22:0/4132624454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdefc0cef70 0x7fdefc0cbae0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009d90 tx=0x7fdef400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfbe81576943fbaa server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:53.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.542+0000 7fdefaffd700 1 -- 172.21.15.22:0/4132624454 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef400e070 con 0x7fdefc0cef70 2023-07-06T01:19:53.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.542+0000 7fdefaffd700 1 -- 172.21.15.22:0/4132624454 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef400bb20 con 0x7fdefc0cef70 2023-07-06T01:19:53.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.542+0000 7fdefaffd700 1 -- 172.21.15.22:0/4132624454 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4012430 con 0x7fdefc0cef70 2023-07-06T01:19:53.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.542+0000 7fdf0297e700 1 -- 172.21.15.22:0/4132624454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdefc0cef70 msgr2=0x7fdefc0cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.542+0000 7fdf0297e700 1 --2- 172.21.15.22:0/4132624454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdefc0cef70 0x7fdefc0cbae0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009d90 tx=0x7fdef400b330 comp rx=0 tx=0).stop 2023-07-06T01:19:53.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.542+0000 7fdf0297e700 1 -- 172.21.15.22:0/4132624454 shutdown_connections 2023-07-06T01:19:53.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.542+0000 7fdf0297e700 1 --2- 172.21.15.22:0/4132624454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdefc0cef70 0x7fdefc0cbae0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.542+0000 7fdf0297e700 1 --2- 172.21.15.22:0/4132624454 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdefc15a710 0x7fdefc0ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.542+0000 7fdf0297e700 1 -- 172.21.15.22:0/4132624454 >> 172.21.15.22:0/4132624454 conn(0x7fdefc0999f0 msgr2=0x7fdefc099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:53.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.542+0000 7fdf0297e700 1 -- 172.21.15.22:0/4132624454 shutdown_connections 2023-07-06T01:19:53.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.542+0000 7fdf0297e700 1 -- 172.21.15.22:0/4132624454 wait complete. 2023-07-06T01:19:53.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.543+0000 7fdf0297e700 1 Processor -- start 2023-07-06T01:19:53.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.544+0000 7fdf0297e700 1 -- start start 2023-07-06T01:19:53.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.544+0000 7fdf0297e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdefc0aecb0 0x7fdefc0ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.544+0000 7fdf0297e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdefc0cef70 0x7fdefc0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.544+0000 7fdf0297e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdefc15a710 0x7fdefc0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.544+0000 7fdf0297e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0b5040 con 0x7fdefc0aecb0 2023-07-06T01:19:53.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.544+0000 7fdf0297e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0c9780 con 0x7fdefc0cef70 2023-07-06T01:19:53.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.544+0000 7fdf0297e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0c9a60 con 0x7fdefc15a710 2023-07-06T01:19:53.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.544+0000 7fdefbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdefc0aecb0 0x7fdefc0ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.544+0000 7fdefbfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdefc0aecb0 0x7fdefc0ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46304/0 (socket says 172.21.15.22:46304) 2023-07-06T01:19:53.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.544+0000 7fdefbfff700 1 -- 172.21.15.22:0/18209394 learned_addr learned my addr 172.21.15.22:0/18209394 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:53.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdefb7fe700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdefc0cef70 0x7fdefc0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdf00f1b700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdefc15a710 0x7fdefc0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdefbfff700 1 -- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdefc15a710 msgr2=0x7fdefc0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdefbfff700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdefc15a710 0x7fdefc0b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdefbfff700 1 -- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdefc0cef70 msgr2=0x7fdefc0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdefbfff700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdefc0cef70 0x7fdefc0b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdefbfff700 1 -- 172.21.15.22:0/18209394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef4009a70 con 0x7fdefc0aecb0 2023-07-06T01:19:53.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdefb7fe700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdefc0cef70 0x7fdefc0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:53.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdefbfff700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdefc0aecb0 0x7fdefc0ca820 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00e870 tx=0x7fdeec00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:53.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdef8ff9700 1 -- 172.21.15.22:0/18209394 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec009590 con 0x7fdefc0aecb0 2023-07-06T01:19:53.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdef8ff9700 1 -- 172.21.15.22:0/18209394 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec00ee70 con 0x7fdefc0aecb0 2023-07-06T01:19:53.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdf0297e700 1 -- 172.21.15.22:0/18209394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc0c9d40 con 0x7fdefc0aecb0 2023-07-06T01:19:53.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdef8ff9700 1 -- 172.21.15.22:0/18209394 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00ca50 con 0x7fdefc0aecb0 2023-07-06T01:19:53.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.545+0000 7fdf0297e700 1 -- 172.21.15.22:0/18209394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc0ca300 con 0x7fdefc0aecb0 2023-07-06T01:19:53.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.546+0000 7fdf0297e700 1 -- 172.21.15.22:0/18209394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc0744b0 con 0x7fdefc0aecb0 2023-07-06T01:19:53.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.547+0000 7fdef8ff9700 1 -- 172.21.15.22:0/18209394 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdeec00cbb0 con 0x7fdefc0aecb0 2023-07-06T01:19:53.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.547+0000 7fdef8ff9700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fdee806ace0 0x7fdee806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.547+0000 7fdef8ff9700 1 -- 172.21.15.22:0/18209394 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdeec0893e0 con 0x7fdefc0aecb0 2023-07-06T01:19:53.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.547+0000 7fdefb7fe700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fdee806ace0 0x7fdee806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.548+0000 7fdefb7fe700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fdee806ace0 0x7fdee806d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdefc1f2ee0 tx=0x7fdef0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:53.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.552+0000 7fdef8ff9700 1 -- 172.21.15.22:0/18209394 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdeec057f20 con 0x7fdefc0aecb0 2023-07-06T01:19:53.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.750+0000 7fdf0297e700 1 -- 172.21.15.22:0/18209394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdefc0ae710 con 0x7fdefc0aecb0 2023-07-06T01:19:53.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.751+0000 7fdef8ff9700 1 -- 172.21.15.22:0/18209394 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7fdeec00ce60 con 0x7fdefc0aecb0 2023-07-06T01:19:53.754 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:53.754 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:53.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.753+0000 7fdf0297e700 1 -- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fdee806ace0 msgr2=0x7fdee806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.753+0000 7fdf0297e700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fdee806ace0 0x7fdee806d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdefc1f2ee0 tx=0x7fdef0009040 comp rx=0 tx=0).stop 2023-07-06T01:19:53.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.753+0000 7fdf0297e700 1 -- 172.21.15.22:0/18209394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdefc0aecb0 msgr2=0x7fdefc0ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.753+0000 7fdf0297e700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdefc0aecb0 0x7fdefc0ca820 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00e870 tx=0x7fdeec00ec30 comp rx=0 tx=0).stop 2023-07-06T01:19:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.753+0000 7fdf0297e700 1 -- 172.21.15.22:0/18209394 shutdown_connections 2023-07-06T01:19:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.753+0000 7fdf0297e700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdefc15a710 0x7fdefc0b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.753+0000 7fdf0297e700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdefc0aecb0 0x7fdefc0ca820 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.753+0000 7fdf0297e700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fdee806ace0 0x7fdee806d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.753+0000 7fdf0297e700 1 --2- 172.21.15.22:0/18209394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdefc0cef70 0x7fdefc0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.753+0000 7fdf0297e700 1 -- 172.21.15.22:0/18209394 >> 172.21.15.22:0/18209394 conn(0x7fdefc0999f0 msgr2=0x7fdefc15e060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:53.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.753+0000 7fdf0297e700 1 -- 172.21.15.22:0/18209394 shutdown_connections 2023-07-06T01:19:53.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.753+0000 7fdf0297e700 1 -- 172.21.15.22:0/18209394 wait complete. 2023-07-06T01:19:53.768 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T01:19:22.873255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T01:19:24.019088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:19:53.769 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:19:53.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.894+0000 7fe99cefe700 1 Processor -- start 2023-07-06T01:19:53.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.894+0000 7fe99cefe700 1 -- start start 2023-07-06T01:19:53.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.895+0000 7fe99cefe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe998159360 0x7fe99815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.895+0000 7fe99cefe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe99806d040 0x7fe99806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.895+0000 7fe99cefe700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9980cf230 con 0x7fe9980cc660 2023-07-06T01:19:53.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.895+0000 7fe99cefe700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99815df20 con 0x7fe99806d040 2023-07-06T01:19:53.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.895+0000 7fe99cefe700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980ce620 con 0x7fe998159360 2023-07-06T01:19:53.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.895+0000 7fe996d9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe99806d040 0x7fe99806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.895+0000 7fe995d9b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe9980cc660 0x7fe9980ccab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37058/0 (socket says 172.21.15.22:37058) 2023-07-06T01:19:53.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.895+0000 7fe995d9b700 1 -- 172.21.15.22:0/1108166099 learned_addr learned my addr 172.21.15.22:0/1108166099 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:53.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.895+0000 7fe99659c700 1 --2- 172.21.15.22:0/1108166099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe998159360 0x7fe99815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.895+0000 7fe996d9d700 1 -- 172.21.15.22:0/1108166099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe998159360 msgr2=0x7fe99815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:53.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.895+0000 7fe996d9d700 1 --2- 172.21.15.22:0/1108166099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe998159360 0x7fe99815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.895+0000 7fe996d9d700 1 -- 172.21.15.22:0/1108166099 >> v1:172.21.15.22:6789/0 conn(0x7fe9980cc660 legacy=0x7fe9980ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.896+0000 7fe996d9d700 1 -- 172.21.15.22:0/1108166099 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9980ce900 con 0x7fe99806d040 2023-07-06T01:19:53.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.895+0000 7fe99559a700 1 -- 172.21.15.22:0/1108166099 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675604705 0 0) 0x7fe9980cf230 con 0x7fe9980cc660 2023-07-06T01:19:53.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.896+0000 7fe99659c700 1 --2- 172.21.15.22:0/1108166099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe998159360 0x7fe99815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:53.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.896+0000 7fe996d9d700 1 --2- 172.21.15.22:0/1108166099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99806d040 0x7fe99806d4a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe990009d90 tx=0x7fe99000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2879b7c9fc6ff84 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:53.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.896+0000 7fe99559a700 1 -- 172.21.15.22:0/1108166099 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99000e070 con 0x7fe99806d040 2023-07-06T01:19:53.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.896+0000 7fe99559a700 1 -- 172.21.15.22:0/1108166099 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99000bbd0 con 0x7fe99806d040 2023-07-06T01:19:53.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.896+0000 7fe99559a700 1 -- 172.21.15.22:0/1108166099 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990012430 con 0x7fe99806d040 2023-07-06T01:19:53.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.896+0000 7fe99cefe700 1 -- 172.21.15.22:0/1108166099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99806d040 msgr2=0x7fe99806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.896+0000 7fe99cefe700 1 --2- 172.21.15.22:0/1108166099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99806d040 0x7fe99806d4a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe990009d90 tx=0x7fe99000b330 comp rx=0 tx=0).stop 2023-07-06T01:19:53.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.896+0000 7fe99cefe700 1 -- 172.21.15.22:0/1108166099 shutdown_connections 2023-07-06T01:19:53.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.896+0000 7fe99cefe700 1 --2- 172.21.15.22:0/1108166099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99806d040 0x7fe99806d4a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.896+0000 7fe99cefe700 1 --2- 172.21.15.22:0/1108166099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe998159360 0x7fe99815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.896+0000 7fe99cefe700 1 -- 172.21.15.22:0/1108166099 >> 172.21.15.22:0/1108166099 conn(0x7fe9980b1890 msgr2=0x7fe9980b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:53.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.897+0000 7fe99cefe700 1 -- 172.21.15.22:0/1108166099 shutdown_connections 2023-07-06T01:19:53.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.897+0000 7fe99cefe700 1 -- 172.21.15.22:0/1108166099 wait complete. 2023-07-06T01:19:53.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.898+0000 7fe99cefe700 1 Processor -- start 2023-07-06T01:19:53.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.898+0000 7fe99cefe700 1 -- start start 2023-07-06T01:19:53.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe99cefe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe99806d040 0x7fe99814da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe99cefe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9980cc660 0x7fe998166e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe99cefe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe998159360 0x7fe998151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe99cefe700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980b51b0 con 0x7fe9980cc660 2023-07-06T01:19:53.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe99cefe700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980b4ed0 con 0x7fe998159360 2023-07-06T01:19:53.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe99cefe700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99815b920 con 0x7fe99806d040 2023-07-06T01:19:53.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe995d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9980cc660 0x7fe998166e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe995d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9980cc660 0x7fe998166e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46320/0 (socket says 172.21.15.22:46320) 2023-07-06T01:19:53.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe995d9b700 1 -- 172.21.15.22:0/1507605639 learned_addr learned my addr 172.21.15.22:0/1507605639 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:53.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe99659c700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe99806d040 0x7fe99814da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe996d9d700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe998159360 0x7fe998151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe995d9b700 1 -- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe99806d040 msgr2=0x7fe99814da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe995d9b700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe99806d040 0x7fe99814da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe995d9b700 1 -- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe998159360 msgr2=0x7fe998151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:53.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe995d9b700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe998159360 0x7fe998151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:53.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe995d9b700 1 -- 172.21.15.22:0/1507605639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe990009a70 con 0x7fe9980cc660 2023-07-06T01:19:53.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe995d9b700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9980cc660 0x7fe998166e90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe9880011e0 tx=0x7fe98800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:53.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe986ffd700 1 -- 172.21.15.22:0/1507605639 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988004c80 con 0x7fe9980cc660 2023-07-06T01:19:53.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe986ffd700 1 -- 172.21.15.22:0/1507605639 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe988004e00 con 0x7fe9980cc660 2023-07-06T01:19:53.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.899+0000 7fe99cefe700 1 -- 172.21.15.22:0/1507605639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99815bc00 con 0x7fe9980cc660 2023-07-06T01:19:53.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.900+0000 7fe986ffd700 1 -- 172.21.15.22:0/1507605639 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988007310 con 0x7fe9980cc660 2023-07-06T01:19:53.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.900+0000 7fe99cefe700 1 -- 172.21.15.22:0/1507605639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99815c400 con 0x7fe9980cc660 2023-07-06T01:19:53.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.901+0000 7fe99cefe700 1 -- 172.21.15.22:0/1507605639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99815b780 con 0x7fe9980cc660 2023-07-06T01:19:53.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.901+0000 7fe986ffd700 1 -- 172.21.15.22:0/1507605639 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe988007470 con 0x7fe9980cc660 2023-07-06T01:19:53.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.902+0000 7fe986ffd700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe97806ace0 0x7fe97806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:53.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.902+0000 7fe986ffd700 1 -- 172.21.15.22:0/1507605639 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9880890d0 con 0x7fe9980cc660 2023-07-06T01:19:53.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.902+0000 7fe99659c700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe97806ace0 0x7fe97806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:53.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.902+0000 7fe99659c700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe97806ace0 0x7fe97806d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe98c006fd0 tx=0x7fe98c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:53.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:53.906+0000 7fe986ffd700 1 -- 172.21.15.22:0/1507605639 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe99815b780 con 0x7fe9980cc660 2023-07-06T01:19:54.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.107+0000 7fe99cefe700 1 -- 172.21.15.22:0/1507605639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe99815c120 con 0x7fe9980cc660 2023-07-06T01:19:54.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.107+0000 7fe986ffd700 1 -- 172.21.15.22:0/1507605639 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe99815c120 con 0x7fe9980cc660 2023-07-06T01:19:54.110 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:19:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.109+0000 7fe99cefe700 1 -- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe97806ace0 msgr2=0x7fe97806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.109+0000 7fe99cefe700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe97806ace0 0x7fe97806d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe98c006fd0 tx=0x7fe98c008040 comp rx=0 tx=0).stop 2023-07-06T01:19:54.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.109+0000 7fe99cefe700 1 -- 172.21.15.22:0/1507605639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9980cc660 msgr2=0x7fe998166e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.109+0000 7fe99cefe700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9980cc660 0x7fe998166e90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe9880011e0 tx=0x7fe98800ca30 comp rx=0 tx=0).stop 2023-07-06T01:19:54.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.109+0000 7fe99cefe700 1 -- 172.21.15.22:0/1507605639 shutdown_connections 2023-07-06T01:19:54.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.109+0000 7fe99cefe700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe97806ace0 0x7fe97806d190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.109+0000 7fe99cefe700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe998159360 0x7fe998151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.109+0000 7fe99cefe700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe99806d040 0x7fe99814da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.109+0000 7fe99cefe700 1 --2- 172.21.15.22:0/1507605639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9980cc660 0x7fe998166e90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.109+0000 7fe99cefe700 1 -- 172.21.15.22:0/1507605639 >> 172.21.15.22:0/1507605639 conn(0x7fe9980b1890 msgr2=0x7fe99806d410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:54.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.110+0000 7fe99cefe700 1 -- 172.21.15.22:0/1507605639 shutdown_connections 2023-07-06T01:19:54.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.110+0000 7fe99cefe700 1 -- 172.21.15.22:0/1507605639 wait complete. 2023-07-06T01:19:54.124 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:19:54.252 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.250+0000 7efdd3e0e700 1 Processor -- start 2023-07-06T01:19:54.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.250+0000 7efdd3e0e700 1 -- start start 2023-07-06T01:19:54.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd3e0e700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7efdcc153480 0x7efdcc15d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd3e0e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efdcc153850 0x7efdcc06a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd3e0e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdcc06aea0 con 0x7efdcc06b320 2023-07-06T01:19:54.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd3e0e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdcc06a8e0 con 0x7efdcc153850 2023-07-06T01:19:54.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd3e0e700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdcc06abc0 con 0x7efdcc153480 2023-07-06T01:19:54.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd13a9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efdcc153850 0x7efdcc06a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd23ab700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7efdcc06b320 0x7efdcc06c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37074/0 (socket says 172.21.15.22:37074) 2023-07-06T01:19:54.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd23ab700 1 -- 172.21.15.22:0/179432161 learned_addr learned my addr 172.21.15.22:0/179432161 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:54.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd1baa700 1 --2- 172.21.15.22:0/179432161 >> v2:172.21.15.103:3300/0 conn(0x7efdcc153480 0x7efdcc15d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd13a9700 1 -- 172.21.15.22:0/179432161 >> v2:172.21.15.103:3300/0 conn(0x7efdcc153480 msgr2=0x7efdcc15d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:54.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd13a9700 1 --2- 172.21.15.22:0/179432161 >> v2:172.21.15.103:3300/0 conn(0x7efdcc153480 0x7efdcc15d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd13a9700 1 -- 172.21.15.22:0/179432161 >> v1:172.21.15.22:6789/0 conn(0x7efdcc06b320 legacy=0x7efdcc06c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd13a9700 1 -- 172.21.15.22:0/179432161 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc15cd80 con 0x7efdcc153850 2023-07-06T01:19:54.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.251+0000 7efdd1baa700 1 --2- 172.21.15.22:0/179432161 >> v2:172.21.15.103:3300/0 conn(0x7efdcc153480 0x7efdcc15d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:54.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.252+0000 7efdd13a9700 1 --2- 172.21.15.22:0/179432161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc153850 0x7efdcc06a000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efdcc06bd00 tx=0x7efdbc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992d136257f56aa3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:54.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.252+0000 7efdd0ba8700 1 -- 172.21.15.22:0/179432161 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc011830 con 0x7efdcc153850 2023-07-06T01:19:54.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.252+0000 7efdd0ba8700 1 -- 172.21.15.22:0/179432161 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdbc011990 con 0x7efdcc153850 2023-07-06T01:19:54.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.252+0000 7efdd0ba8700 1 -- 172.21.15.22:0/179432161 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc011d20 con 0x7efdcc153850 2023-07-06T01:19:54.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.252+0000 7efdd3e0e700 1 -- 172.21.15.22:0/179432161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc153850 msgr2=0x7efdcc06a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.252+0000 7efdd3e0e700 1 --2- 172.21.15.22:0/179432161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc153850 0x7efdcc06a000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efdcc06bd00 tx=0x7efdbc009b20 comp rx=0 tx=0).stop 2023-07-06T01:19:54.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.252+0000 7efdd3e0e700 1 -- 172.21.15.22:0/179432161 shutdown_connections 2023-07-06T01:19:54.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.252+0000 7efdd3e0e700 1 --2- 172.21.15.22:0/179432161 >> v2:172.21.15.103:3300/0 conn(0x7efdcc153480 0x7efdcc15d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.252+0000 7efdd3e0e700 1 --2- 172.21.15.22:0/179432161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc153850 0x7efdcc06a000 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.252+0000 7efdd3e0e700 1 -- 172.21.15.22:0/179432161 >> 172.21.15.22:0/179432161 conn(0x7efdcc0ccb00 msgr2=0x7efdcc0cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.252+0000 7efdd3e0e700 1 -- 172.21.15.22:0/179432161 shutdown_connections 2023-07-06T01:19:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.252+0000 7efdd3e0e700 1 -- 172.21.15.22:0/179432161 wait complete. 2023-07-06T01:19:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.254+0000 7efdd3e0e700 1 Processor -- start 2023-07-06T01:19:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.254+0000 7efdd3e0e700 1 -- start start 2023-07-06T01:19:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.254+0000 7efdd3e0e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdcc06b320 0x7efdcc0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.254+0000 7efdd3e0e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdcc153480 0x7efdcc152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.254+0000 7efdd3e0e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc153850 0x7efdcc071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.254+0000 7efdd3e0e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc0b36a0 con 0x7efdcc153850 2023-07-06T01:19:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.254+0000 7efdd3e0e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc15ea40 con 0x7efdcc06b320 2023-07-06T01:19:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.254+0000 7efdd3e0e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc15ed20 con 0x7efdcc153480 2023-07-06T01:19:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.254+0000 7efdd23ab700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc153850 0x7efdcc071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.254+0000 7efdd23ab700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc153850 0x7efdcc071310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46336/0 (socket says 172.21.15.22:46336) 2023-07-06T01:19:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.254+0000 7efdd23ab700 1 -- 172.21.15.22:0/2724089141 learned_addr learned my addr 172.21.15.22:0/2724089141 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdd13a9700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdcc153480 0x7efdcc152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdd1baa700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdcc06b320 0x7efdcc0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdd23ab700 1 -- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdcc153480 msgr2=0x7efdcc152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdd23ab700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdcc153480 0x7efdcc152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdd23ab700 1 -- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdcc06b320 msgr2=0x7efdcc0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdd23ab700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdcc06b320 0x7efdcc0b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdd23ab700 1 -- 172.21.15.22:0/2724089141 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc0094a0 con 0x7efdcc153850 2023-07-06T01:19:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdd1baa700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdcc06b320 0x7efdcc0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdd23ab700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc153850 0x7efdcc071310 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efdc400c500 tx=0x7efdc400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdc27fc700 1 -- 172.21.15.22:0/2724089141 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4005d70 con 0x7efdcc153850 2023-07-06T01:19:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdc27fc700 1 -- 172.21.15.22:0/2724089141 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc40070d0 con 0x7efdcc153850 2023-07-06T01:19:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdd3e0e700 1 -- 172.21.15.22:0/2724089141 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdcc15f000 con 0x7efdcc153850 2023-07-06T01:19:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdc27fc700 1 -- 172.21.15.22:0/2724089141 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4007440 con 0x7efdcc153850 2023-07-06T01:19:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.255+0000 7efdd3e0e700 1 -- 172.21.15.22:0/2724089141 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdcc15f5c0 con 0x7efdcc153850 2023-07-06T01:19:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.256+0000 7efdd3e0e700 1 -- 172.21.15.22:0/2724089141 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdcc0744b0 con 0x7efdcc153850 2023-07-06T01:19:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.257+0000 7efdc27fc700 1 -- 172.21.15.22:0/2724089141 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efdc40075a0 con 0x7efdcc153850 2023-07-06T01:19:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.257+0000 7efdc27fc700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efdb006ace0 0x7efdb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.257+0000 7efdc27fc700 1 -- 172.21.15.22:0/2724089141 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efdc4088840 con 0x7efdcc153850 2023-07-06T01:19:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.257+0000 7efdd1baa700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efdb006ace0 0x7efdb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.258+0000 7efdd1baa700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efdb006ace0 0x7efdb006d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efdc800d430 tx=0x7efdc8007530 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:54.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.261+0000 7efdc27fc700 1 -- 172.21.15.22:0/2724089141 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efdc40539a0 con 0x7efdcc153850 2023-07-06T01:19:54.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.459+0000 7efdd3e0e700 1 -- 172.21.15.22:0/2724089141 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efdcc0cfd60 con 0x7efdcc153850 2023-07-06T01:19:54.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.459+0000 7efdc27fc700 1 -- 172.21.15.22:0/2724089141 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7efdc4056fc0 con 0x7efdcc153850 2023-07-06T01:19:54.463 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:19:54.464 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.462+0000 7efdd3e0e700 1 -- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efdb006ace0 msgr2=0x7efdb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.462+0000 7efdd3e0e700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efdb006ace0 0x7efdb006d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efdc800d430 tx=0x7efdc8007530 comp rx=0 tx=0).stop 2023-07-06T01:19:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.462+0000 7efdd3e0e700 1 -- 172.21.15.22:0/2724089141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc153850 msgr2=0x7efdcc071310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.462+0000 7efdd3e0e700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc153850 0x7efdcc071310 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efdc400c500 tx=0x7efdc400c8c0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.462+0000 7efdd3e0e700 1 -- 172.21.15.22:0/2724089141 shutdown_connections 2023-07-06T01:19:54.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.462+0000 7efdd3e0e700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efdcc153850 0x7efdcc071310 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.462+0000 7efdd3e0e700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efdb006ace0 0x7efdb006d190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.462+0000 7efdd3e0e700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdcc06b320 0x7efdcc0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.462+0000 7efdd3e0e700 1 --2- 172.21.15.22:0/2724089141 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdcc153480 0x7efdcc152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.462+0000 7efdd3e0e700 1 -- 172.21.15.22:0/2724089141 >> 172.21.15.22:0/2724089141 conn(0x7efdcc0ccb00 msgr2=0x7efdcc06c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.462+0000 7efdd3e0e700 1 -- 172.21.15.22:0/2724089141 shutdown_connections 2023-07-06T01:19:54.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.462+0000 7efdd3e0e700 1 -- 172.21.15.22:0/2724089141 wait complete. 2023-07-06T01:19:54.476 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T01:19:54.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.602+0000 7f86cbee2700 1 Processor -- start 2023-07-06T01:19:54.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.602+0000 7f86cbee2700 1 -- start start 2023-07-06T01:19:54.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86cbee2700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c406a710 0x7f86c406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86cbee2700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f86c40cbc10 0x7f86c40cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86cbee2700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c40af670 con 0x7f86c40af2a0 2023-07-06T01:19:54.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86cbee2700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c40cd560 con 0x7f86c40cbc10 2023-07-06T01:19:54.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86cbee2700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c415cab0 con 0x7f86c406a710 2023-07-06T01:19:54.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86c9c7e700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f86c40af2a0 0x7f86c406a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37084/0 (socket says 172.21.15.22:37084) 2023-07-06T01:19:54.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86c9c7e700 1 -- 172.21.15.22:0/2474707811 learned_addr learned my addr 172.21.15.22:0/2474707811 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:54.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86ca47f700 1 --2- 172.21.15.22:0/2474707811 >> v2:172.21.15.22:3300/0 conn(0x7f86c40cbc10 0x7f86c40cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86c947d700 1 --2- 172.21.15.22:0/2474707811 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c406a710 0x7f86c406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86c8c7c700 1 -- 172.21.15.22:0/2474707811 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011344320 0 0) 0x7f86c40af670 con 0x7f86c40af2a0 2023-07-06T01:19:54.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86c8c7c700 1 -- 172.21.15.22:0/2474707811 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86b00035b0 con 0x7f86c40af2a0 2023-07-06T01:19:54.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86ca47f700 1 -- 172.21.15.22:0/2474707811 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c406a710 msgr2=0x7f86c406caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:54.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86ca47f700 1 --2- 172.21.15.22:0/2474707811 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c406a710 0x7f86c406caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86ca47f700 1 -- 172.21.15.22:0/2474707811 >> v1:172.21.15.22:6789/0 conn(0x7f86c40af2a0 legacy=0x7f86c406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.603+0000 7f86ca47f700 1 -- 172.21.15.22:0/2474707811 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c406d030 con 0x7f86c40cbc10 2023-07-06T01:19:54.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.604+0000 7f86ca47f700 1 --2- 172.21.15.22:0/2474707811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40cbc10 0x7f86c40cc050 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009d90 tx=0x7f86bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c3d09a0a5bcfb0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:54.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.604+0000 7f86c8c7c700 1 -- 172.21.15.22:0/2474707811 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc00e070 con 0x7f86c40cbc10 2023-07-06T01:19:54.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.604+0000 7f86c8c7c700 1 -- 172.21.15.22:0/2474707811 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc00bb20 con 0x7f86c40cbc10 2023-07-06T01:19:54.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.604+0000 7f86c8c7c700 1 -- 172.21.15.22:0/2474707811 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc012430 con 0x7f86c40cbc10 2023-07-06T01:19:54.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.604+0000 7f86cbee2700 1 -- 172.21.15.22:0/2474707811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40cbc10 msgr2=0x7f86c40cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.604+0000 7f86cbee2700 1 --2- 172.21.15.22:0/2474707811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40cbc10 0x7f86c40cc050 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009d90 tx=0x7f86bc00b330 comp rx=0 tx=0).stop 2023-07-06T01:19:54.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.604+0000 7f86cbee2700 1 -- 172.21.15.22:0/2474707811 shutdown_connections 2023-07-06T01:19:54.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.604+0000 7f86cbee2700 1 --2- 172.21.15.22:0/2474707811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40cbc10 0x7f86c40cc050 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.604+0000 7f86cbee2700 1 --2- 172.21.15.22:0/2474707811 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c406a710 0x7f86c406caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.604+0000 7f86cbee2700 1 -- 172.21.15.22:0/2474707811 >> 172.21.15.22:0/2474707811 conn(0x7f86c40b1890 msgr2=0x7f86c40b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:54.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.604+0000 7f86cbee2700 1 -- 172.21.15.22:0/2474707811 shutdown_connections 2023-07-06T01:19:54.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.604+0000 7f86cbee2700 1 -- 172.21.15.22:0/2474707811 wait complete. 2023-07-06T01:19:54.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.606+0000 7f86cbee2700 1 Processor -- start 2023-07-06T01:19:54.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.606+0000 7f86cbee2700 1 -- start start 2023-07-06T01:19:54.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86cbee2700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86c406a710 0x7f86c415f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86cbee2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40af2a0 0x7f86c4151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86cbee2700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c40cbc10 0x7f86c415de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86cbee2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c415fd50 con 0x7f86c40af2a0 2023-07-06T01:19:54.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86cbee2700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c415f380 con 0x7f86c406a710 2023-07-06T01:19:54.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86cbee2700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c415e770 con 0x7f86c40cbc10 2023-07-06T01:19:54.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86c947d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40af2a0 0x7f86c4151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86c947d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40af2a0 0x7f86c4151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46348/0 (socket says 172.21.15.22:46348) 2023-07-06T01:19:54.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86c947d700 1 -- 172.21.15.22:0/1443502377 learned_addr learned my addr 172.21.15.22:0/1443502377 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:54.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86c9c7e700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86c406a710 0x7f86c415f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86ca47f700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c40cbc10 0x7f86c415de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86c947d700 1 -- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c40cbc10 msgr2=0x7f86c415de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86c947d700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c40cbc10 0x7f86c415de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86c947d700 1 -- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86c406a710 msgr2=0x7f86c415f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86c947d700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86c406a710 0x7f86c415f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.607+0000 7f86c947d700 1 -- 172.21.15.22:0/1443502377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc009a70 con 0x7f86c40af2a0 2023-07-06T01:19:54.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.608+0000 7f86c947d700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40af2a0 0x7f86c4151570 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f86c000e870 tx=0x7f86c000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:54.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.608+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1443502377 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0009590 con 0x7f86c40af2a0 2023-07-06T01:19:54.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.608+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1443502377 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c0013680 con 0x7f86c40af2a0 2023-07-06T01:19:54.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.608+0000 7f86cbee2700 1 -- 172.21.15.22:0/1443502377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c415ea50 con 0x7f86c40af2a0 2023-07-06T01:19:54.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.608+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1443502377 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c00139f0 con 0x7f86c40af2a0 2023-07-06T01:19:54.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.608+0000 7f86cbee2700 1 -- 172.21.15.22:0/1443502377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c415f010 con 0x7f86c40af2a0 2023-07-06T01:19:54.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.609+0000 7f86cbee2700 1 -- 172.21.15.22:0/1443502377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c415f4f0 con 0x7f86c40af2a0 2023-07-06T01:19:54.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.609+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1443502377 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f86c0013b50 con 0x7f86c40af2a0 2023-07-06T01:19:54.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.609+0000 7f86ba7fc700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f86a806ace0 0x7f86a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.610+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1443502377 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f86c00891f0 con 0x7f86c40af2a0 2023-07-06T01:19:54.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.610+0000 7f86c9c7e700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f86a806ace0 0x7f86a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.610+0000 7f86c9c7e700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f86a806ace0 0x7f86a806d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f86b4007fd0 tx=0x7f86b4007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:54.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.614+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1443502377 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f86c0057dc0 con 0x7f86c40af2a0 2023-07-06T01:19:54.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.813+0000 7f86cbee2700 1 -- 172.21.15.22:0/1443502377 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f86c40744b0 con 0x7f86c40af2a0 2023-07-06T01:19:54.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.813+0000 7f86ba7fc700 1 -- 172.21.15.22:0/1443502377 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f86c0057950 con 0x7f86c40af2a0 2023-07-06T01:19:54.816 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-07-06T01:19:54.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.816+0000 7f86cbee2700 1 -- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f86a806ace0 msgr2=0x7f86a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.816+0000 7f86cbee2700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f86a806ace0 0x7f86a806d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f86b4007fd0 tx=0x7f86b4007dc0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.816+0000 7f86cbee2700 1 -- 172.21.15.22:0/1443502377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40af2a0 msgr2=0x7f86c4151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.816+0000 7f86cbee2700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40af2a0 0x7f86c4151570 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f86c000e870 tx=0x7f86c000eb80 comp rx=0 tx=0).stop 2023-07-06T01:19:54.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.816+0000 7f86cbee2700 1 -- 172.21.15.22:0/1443502377 shutdown_connections 2023-07-06T01:19:54.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.816+0000 7f86cbee2700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c40cbc10 0x7f86c415de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.816+0000 7f86cbee2700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f86a806ace0 0x7f86a806d190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.816+0000 7f86cbee2700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86c406a710 0x7f86c415f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.816+0000 7f86cbee2700 1 --2- 172.21.15.22:0/1443502377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86c40af2a0 0x7f86c4151570 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.816+0000 7f86cbee2700 1 -- 172.21.15.22:0/1443502377 >> 172.21.15.22:0/1443502377 conn(0x7f86c40b1890 msgr2=0x7f86c406a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:54.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.816+0000 7f86cbee2700 1 -- 172.21.15.22:0/1443502377 shutdown_connections 2023-07-06T01:19:54.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.816+0000 7f86cbee2700 1 -- 172.21.15.22:0/1443502377 wait complete. 2023-07-06T01:19:54.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T01:19:54.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.955+0000 7f1cb1234700 1 Processor -- start 2023-07-06T01:19:54.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.955+0000 7f1cb1234700 1 -- start start 2023-07-06T01:19:54.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.956+0000 7f1cb1234700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1cac0ce860 0x7f1cac0cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.956+0000 7f1cb1234700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1cac0cbae0 0x7f1cac0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.956+0000 7f1cb1234700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac0af270 con 0x7f1cac15a710 2023-07-06T01:19:54.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.956+0000 7f1cb1234700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cac0cf210 con 0x7f1cac0ce860 2023-07-06T01:19:54.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.956+0000 7f1cb1234700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0af550 con 0x7f1cac0cbae0 2023-07-06T01:19:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.956+0000 7f1caa59c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1cac0ce860 0x7f1cac0cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.956+0000 7f1caa59c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1cac0ce860 0x7f1cac0cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46356/0 (socket says 172.21.15.22:46356) 2023-07-06T01:19:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.956+0000 7f1caa59c700 1 -- 172.21.15.22:0/2997483766 learned_addr learned my addr 172.21.15.22:0/2997483766 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.956+0000 7f1cab59e700 1 --2- 172.21.15.22:0/2997483766 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1cac0cbae0 0x7f1cac0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.956+0000 7f1caa59c700 1 -- 172.21.15.22:0/2997483766 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1cac0cbae0 msgr2=0x7f1cac0cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.956+0000 7f1caa59c700 1 --2- 172.21.15.22:0/2997483766 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1cac0cbae0 0x7f1cac0cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.956+0000 7f1caa59c700 1 -- 172.21.15.22:0/2997483766 >> v1:172.21.15.103:6789/0 conn(0x7f1cac15a710 legacy=0x7f1cac0ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:54.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.956+0000 7f1caa59c700 1 -- 172.21.15.22:0/2997483766 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac0af830 con 0x7f1cac0ce860 2023-07-06T01:19:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.957+0000 7f1caa59c700 1 --2- 172.21.15.22:0/2997483766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0ce860 0x7f1cac0cecd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0098f0 tx=0x7f1c9c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a91f025fc14928b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.957+0000 7f1ca9d9b700 1 -- 172.21.15.22:0/2997483766 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c004230 con 0x7f1cac0ce860 2023-07-06T01:19:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.957+0000 7f1ca9d9b700 1 -- 172.21.15.22:0/2997483766 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c004390 con 0x7f1cac0ce860 2023-07-06T01:19:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.957+0000 7f1ca9d9b700 1 -- 172.21.15.22:0/2997483766 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0036a0 con 0x7f1cac0ce860 2023-07-06T01:19:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.957+0000 7f1cb1234700 1 -- 172.21.15.22:0/2997483766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0ce860 msgr2=0x7f1cac0cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.957+0000 7f1cb1234700 1 --2- 172.21.15.22:0/2997483766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0ce860 0x7f1cac0cecd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0098f0 tx=0x7f1c9c009c00 comp rx=0 tx=0).stop 2023-07-06T01:19:54.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.957+0000 7f1cb1234700 1 -- 172.21.15.22:0/2997483766 shutdown_connections 2023-07-06T01:19:54.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.957+0000 7f1cb1234700 1 --2- 172.21.15.22:0/2997483766 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1cac0cbae0 0x7f1cac0cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.957+0000 7f1cb1234700 1 --2- 172.21.15.22:0/2997483766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0ce860 0x7f1cac0cecd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.957+0000 7f1cb1234700 1 -- 172.21.15.22:0/2997483766 >> 172.21.15.22:0/2997483766 conn(0x7f1cac0999f0 msgr2=0x7f1cac099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:54.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.957+0000 7f1cb1234700 1 -- 172.21.15.22:0/2997483766 shutdown_connections 2023-07-06T01:19:54.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.957+0000 7f1cb1234700 1 -- 172.21.15.22:0/2997483766 wait complete. 2023-07-06T01:19:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.959+0000 7f1cb1234700 1 Processor -- start 2023-07-06T01:19:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.959+0000 7f1cb1234700 1 -- start start 2023-07-06T01:19:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.959+0000 7f1cb1234700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0cbae0 0x7f1cac0c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1cb1234700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1cac0ce860 0x7f1cac156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1cb1234700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cac15a710 0x7f1cac153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1cb1234700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0b5d70 con 0x7f1cac0cbae0 2023-07-06T01:19:54.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1cb1234700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0b4cd0 con 0x7f1cac15a710 2023-07-06T01:19:54.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1cb1234700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0b4fb0 con 0x7f1cac0ce860 2023-07-06T01:19:54.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1caad9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0cbae0 0x7f1cac0c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1caad9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0cbae0 0x7f1cac0c6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46360/0 (socket says 172.21.15.22:46360) 2023-07-06T01:19:54.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1caad9d700 1 -- 172.21.15.22:0/963644108 learned_addr learned my addr 172.21.15.22:0/963644108 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:54.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1cab59e700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cac15a710 0x7f1cac153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1caa59c700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1cac0ce860 0x7f1cac156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1caad9d700 1 -- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1cac0ce860 msgr2=0x7f1cac156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1caad9d700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1cac0ce860 0x7f1cac156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1caad9d700 1 -- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cac15a710 msgr2=0x7f1cac153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:54.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1caad9d700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cac15a710 0x7f1cac153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:54.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1caad9d700 1 -- 172.21.15.22:0/963644108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c009550 con 0x7f1cac0cbae0 2023-07-06T01:19:54.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1caa59c700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1cac0ce860 0x7f1cac156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1cab59e700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cac15a710 0x7f1cac153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1caad9d700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0cbae0 0x7f1cac0c6e80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000c4d0 tx=0x7f1ca000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1c937fe700 1 -- 172.21.15.22:0/963644108 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0011790 con 0x7f1cac0cbae0 2023-07-06T01:19:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1c937fe700 1 -- 172.21.15.22:0/963644108 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca00118f0 con 0x7f1cac0cbae0 2023-07-06T01:19:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1cb1234700 1 -- 172.21.15.22:0/963644108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac0b5290 con 0x7f1cac0cbae0 2023-07-06T01:19:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1c937fe700 1 -- 172.21.15.22:0/963644108 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0011ca0 con 0x7f1cac0cbae0 2023-07-06T01:19:54.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.960+0000 7f1cb1234700 1 -- 172.21.15.22:0/963644108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac0b5850 con 0x7f1cac0cbae0 2023-07-06T01:19:54.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.961+0000 7f1cb1234700 1 -- 172.21.15.22:0/963644108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac06fba0 con 0x7f1cac0cbae0 2023-07-06T01:19:54.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.962+0000 7f1c937fe700 1 -- 172.21.15.22:0/963644108 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1ca001c460 con 0x7f1cac0cbae0 2023-07-06T01:19:54.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.962+0000 7f1c937fe700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1c9806ace0 0x7f1c9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:54.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.962+0000 7f1c937fe700 1 -- 172.21.15.22:0/963644108 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1ca00892d0 con 0x7f1cac0cbae0 2023-07-06T01:19:54.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.963+0000 7f1caa59c700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1c9806ace0 0x7f1c9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:54.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.963+0000 7f1caa59c700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1c9806ace0 0x7f1c9806d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009130 tx=0x7f1c9c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:54.970 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:54.967+0000 7f1c937fe700 1 -- 172.21.15.22:0/963644108 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1ca0054370 con 0x7f1cac0cbae0 2023-07-06T01:19:55.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.164+0000 7f1cb1234700 1 -- 172.21.15.22:0/963644108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1cac0cfd10 con 0x7f1cac0cbae0 2023-07-06T01:19:55.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.164+0000 7f1c937fe700 1 -- 172.21.15.22:0/963644108 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1ca0057990 con 0x7f1cac0cbae0 2023-07-06T01:19:55.167 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-07-06T01:19:55.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.166+0000 7f1cb1234700 1 -- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1c9806ace0 msgr2=0x7f1c9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.166+0000 7f1cb1234700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1c9806ace0 0x7f1c9806d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009130 tx=0x7f1c9c008030 comp rx=0 tx=0).stop 2023-07-06T01:19:55.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.166+0000 7f1cb1234700 1 -- 172.21.15.22:0/963644108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0cbae0 msgr2=0x7f1cac0c6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.166+0000 7f1cb1234700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0cbae0 0x7f1cac0c6e80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000c4d0 tx=0x7f1ca000c890 comp rx=0 tx=0).stop 2023-07-06T01:19:55.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.167+0000 7f1cb1234700 1 -- 172.21.15.22:0/963644108 shutdown_connections 2023-07-06T01:19:55.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.167+0000 7f1cb1234700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1c9806ace0 0x7f1c9806d190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.167+0000 7f1cb1234700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1cac15a710 0x7f1cac153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.167+0000 7f1cb1234700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1cac0cbae0 0x7f1cac0c6e80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.167+0000 7f1cb1234700 1 --2- 172.21.15.22:0/963644108 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1cac0ce860 0x7f1cac156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.167+0000 7f1cb1234700 1 -- 172.21.15.22:0/963644108 >> 172.21.15.22:0/963644108 conn(0x7f1cac0999f0 msgr2=0x7f1cac157160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:55.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.167+0000 7f1cb1234700 1 -- 172.21.15.22:0/963644108 shutdown_connections 2023-07-06T01:19:55.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.167+0000 7f1cb1234700 1 -- 172.21.15.22:0/963644108 wait complete. 2023-07-06T01:19:55.182 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:55.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.305+0000 7efed0b6e700 1 Processor -- start 2023-07-06T01:19:55.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.305+0000 7efed0b6e700 1 -- start start 2023-07-06T01:19:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efed0b6e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efecc06a710 0x7efecc06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efed0b6e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efecc0cbc10 0x7efecc0cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efed0b6e700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc0af670 con 0x7efecc0af2a0 2023-07-06T01:19:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efed0b6e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efecc0cd560 con 0x7efecc06a710 2023-07-06T01:19:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efed0b6e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc15cab0 con 0x7efecc0cbc10 2023-07-06T01:19:55.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efec9d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efecc06a710 0x7efecc06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efec9d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efecc06a710 0x7efecc06caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46370/0 (socket says 172.21.15.22:46370) 2023-07-06T01:19:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efec9d9b700 1 -- 172.21.15.22:0/1464657293 learned_addr learned my addr 172.21.15.22:0/1464657293 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efecad9d700 1 --2- 172.21.15.22:0/1464657293 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efecc0cbc10 0x7efecc0cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efec9d9b700 1 -- 172.21.15.22:0/1464657293 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efecc0cbc10 msgr2=0x7efecc0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efec9d9b700 1 --2- 172.21.15.22:0/1464657293 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efecc0cbc10 0x7efecc0cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efec9d9b700 1 -- 172.21.15.22:0/1464657293 >> v1:172.21.15.103:6789/0 conn(0x7efecc0af2a0 legacy=0x7efecc06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efec9d9b700 1 -- 172.21.15.22:0/1464657293 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc06d030 con 0x7efecc06a710 2023-07-06T01:19:55.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efecad9d700 1 --2- 172.21.15.22:0/1464657293 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efecc0cbc10 0x7efecc0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:55.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efec9d9b700 1 --2- 172.21.15.22:0/1464657293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efecc06a710 0x7efecc06caf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efec00098f0 tx=0x7efec0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963594945687560c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:55.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efec959a700 1 -- 172.21.15.22:0/1464657293 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000b8a0 con 0x7efecc06a710 2023-07-06T01:19:55.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efec959a700 1 -- 172.21.15.22:0/1464657293 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec000ba00 con 0x7efecc06a710 2023-07-06T01:19:55.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.306+0000 7efec959a700 1 -- 172.21.15.22:0/1464657293 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000bd90 con 0x7efecc06a710 2023-07-06T01:19:55.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.307+0000 7efed0b6e700 1 -- 172.21.15.22:0/1464657293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efecc06a710 msgr2=0x7efecc06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.307+0000 7efed0b6e700 1 --2- 172.21.15.22:0/1464657293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efecc06a710 0x7efecc06caf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efec00098f0 tx=0x7efec0009c00 comp rx=0 tx=0).stop 2023-07-06T01:19:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.307+0000 7efed0b6e700 1 -- 172.21.15.22:0/1464657293 shutdown_connections 2023-07-06T01:19:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.307+0000 7efed0b6e700 1 --2- 172.21.15.22:0/1464657293 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efecc0cbc10 0x7efecc0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.307+0000 7efed0b6e700 1 --2- 172.21.15.22:0/1464657293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efecc06a710 0x7efecc06caf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.307+0000 7efed0b6e700 1 -- 172.21.15.22:0/1464657293 >> 172.21.15.22:0/1464657293 conn(0x7efecc0b1890 msgr2=0x7efecc0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.307+0000 7efed0b6e700 1 -- 172.21.15.22:0/1464657293 shutdown_connections 2023-07-06T01:19:55.313 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.307+0000 7efed0b6e700 1 -- 172.21.15.22:0/1464657293 wait complete. 2023-07-06T01:19:55.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.308+0000 7efed0b6e700 1 Processor -- start 2023-07-06T01:19:55.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.308+0000 7efed0b6e700 1 -- start start 2023-07-06T01:19:55.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.309+0000 7efed0b6e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efecc06a710 0x7efecc14e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:55.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.309+0000 7efed0b6e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efecc0af2a0 0x7efecc0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:55.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.309+0000 7efed0b6e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efecc0cbc10 0x7efecc0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:55.316 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.309+0000 7efed0b6e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0b3c00 con 0x7efecc0af2a0 2023-07-06T01:19:55.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.309+0000 7efed0b6e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc15d850 con 0x7efecc0cbc10 2023-07-06T01:19:55.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.309+0000 7efed0b6e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc15e3d0 con 0x7efecc06a710 2023-07-06T01:19:55.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.309+0000 7efec9d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efecc0af2a0 0x7efecc0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:55.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.309+0000 7efec9d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efecc0af2a0 0x7efecc0b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46376/0 (socket says 172.21.15.22:46376) 2023-07-06T01:19:55.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.309+0000 7efec9d9b700 1 -- 172.21.15.22:0/2682074485 learned_addr learned my addr 172.21.15.22:0/2682074485 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:55.317 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.309+0000 7efecad9d700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efecc0cbc10 0x7efecc0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.309+0000 7efeca59c700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efecc06a710 0x7efecc14e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.309+0000 7efec9d9b700 1 -- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efecc06a710 msgr2=0x7efecc14e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.309+0000 7efec9d9b700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efecc06a710 0x7efecc14e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.310+0000 7efec9d9b700 1 -- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efecc0cbc10 msgr2=0x7efecc0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.310+0000 7efec9d9b700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efecc0cbc10 0x7efecc0b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.310+0000 7efec9d9b700 1 -- 172.21.15.22:0/2682074485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec0009550 con 0x7efecc0af2a0 2023-07-06T01:19:55.318 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.310+0000 7efecad9d700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efecc0cbc10 0x7efecc0b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.310+0000 7efec9d9b700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efecc0af2a0 0x7efecc0b36c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efec0009c40 tx=0x7efec0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.310+0000 7efebaffd700 1 -- 172.21.15.22:0/2682074485 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0004480 con 0x7efecc0af2a0 2023-07-06T01:19:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.310+0000 7efebaffd700 1 -- 172.21.15.22:0/2682074485 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec001f070 con 0x7efecc0af2a0 2023-07-06T01:19:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.310+0000 7efed0b6e700 1 -- 172.21.15.22:0/2682074485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc0b30d0 con 0x7efecc0af2a0 2023-07-06T01:19:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.310+0000 7efebaffd700 1 -- 172.21.15.22:0/2682074485 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00038c0 con 0x7efecc0af2a0 2023-07-06T01:19:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.310+0000 7efed0b6e700 1 -- 172.21.15.22:0/2682074485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc15dd80 con 0x7efecc0af2a0 2023-07-06T01:19:55.319 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.311+0000 7efed0b6e700 1 -- 172.21.15.22:0/2682074485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc066070 con 0x7efecc0af2a0 2023-07-06T01:19:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.311+0000 7efebaffd700 1 -- 172.21.15.22:0/2682074485 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efec0021030 con 0x7efecc0af2a0 2023-07-06T01:19:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.312+0000 7efebaffd700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efeac06ace0 0x7efeac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.312+0000 7efebaffd700 1 -- 172.21.15.22:0/2682074485 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efec008a7f0 con 0x7efecc0af2a0 2023-07-06T01:19:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.312+0000 7efeca59c700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efeac06ace0 0x7efeac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:55.320 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.312+0000 7efeca59c700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efeac06ace0 0x7efeac06d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efecc15bec0 tx=0x7efebc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:55.321 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.316+0000 7efebaffd700 1 -- 172.21.15.22:0/2682074485 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efec0055a50 con 0x7efecc0af2a0 2023-07-06T01:19:55.518 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.516+0000 7efed0b6e700 1 -- 172.21.15.22:0/2682074485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efecc0744b0 con 0x7efecc0af2a0 2023-07-06T01:19:55.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.517+0000 7efebaffd700 1 -- 172.21.15.22:0/2682074485 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7efec0059070 con 0x7efecc0af2a0 2023-07-06T01:19:55.520 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:55.520 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:55.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.519+0000 7efed0b6e700 1 -- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efeac06ace0 msgr2=0x7efeac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.519+0000 7efed0b6e700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efeac06ace0 0x7efeac06d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efecc15bec0 tx=0x7efebc009040 comp rx=0 tx=0).stop 2023-07-06T01:19:55.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.519+0000 7efed0b6e700 1 -- 172.21.15.22:0/2682074485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efecc0af2a0 msgr2=0x7efecc0b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.522 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.519+0000 7efed0b6e700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efecc0af2a0 0x7efecc0b36c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efec0009c40 tx=0x7efec0004750 comp rx=0 tx=0).stop 2023-07-06T01:19:55.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.519+0000 7efed0b6e700 1 -- 172.21.15.22:0/2682074485 shutdown_connections 2023-07-06T01:19:55.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.519+0000 7efed0b6e700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7efeac06ace0 0x7efeac06d190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.519+0000 7efed0b6e700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efecc0cbc10 0x7efecc0b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.519+0000 7efed0b6e700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efecc06a710 0x7efecc14e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.519+0000 7efed0b6e700 1 --2- 172.21.15.22:0/2682074485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efecc0af2a0 0x7efecc0b36c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.519+0000 7efed0b6e700 1 -- 172.21.15.22:0/2682074485 >> 172.21.15.22:0/2682074485 conn(0x7efecc0b1890 msgr2=0x7efecc0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:55.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.519+0000 7efed0b6e700 1 -- 172.21.15.22:0/2682074485 shutdown_connections 2023-07-06T01:19:55.524 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.519+0000 7efed0b6e700 1 -- 172.21.15.22:0/2682074485 wait complete. 2023-07-06T01:19:55.534 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T01:19:22.873255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T01:19:24.019088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:19:55.535 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:19:55.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.662+0000 7f6c35491700 1 Processor -- start 2023-07-06T01:19:55.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.662+0000 7f6c35491700 1 -- start start 2023-07-06T01:19:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c35491700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f6c300b3b70 0x7f6c300d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c35491700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6c300b40c0 0x7f6c300d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c35491700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c300cebd0 con 0x7f6c300b36f0 2023-07-06T01:19:55.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c35491700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3015cff0 con 0x7f6c300b40c0 2023-07-06T01:19:55.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c35491700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c300cdfc0 con 0x7f6c300b3b70 2023-07-06T01:19:55.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c2f7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6c300b40c0 0x7f6c300d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:55.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c2f7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6c300b40c0 0x7f6c300d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46386/0 (socket says 172.21.15.22:46386) 2023-07-06T01:19:55.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c2f7fe700 1 -- 172.21.15.22:0/621353136 learned_addr learned my addr 172.21.15.22:0/621353136 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:55.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c2e7fc700 1 --2- 172.21.15.22:0/621353136 >> v2:172.21.15.103:3300/0 conn(0x7f6c300b3b70 0x7f6c300d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:55.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c2f7fe700 1 -- 172.21.15.22:0/621353136 >> v2:172.21.15.103:3300/0 conn(0x7f6c300b3b70 msgr2=0x7f6c300d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:55.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c2f7fe700 1 --2- 172.21.15.22:0/621353136 >> v2:172.21.15.103:3300/0 conn(0x7f6c300b3b70 0x7f6c300d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c2f7fe700 1 -- 172.21.15.22:0/621353136 >> v1:172.21.15.103:6789/0 conn(0x7f6c300b36f0 legacy=0x7f6c300cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c2f7fe700 1 -- 172.21.15.22:0/621353136 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c300ce2a0 con 0x7f6c300b40c0 2023-07-06T01:19:55.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c2f7fe700 1 --2- 172.21.15.22:0/621353136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c300b40c0 0x7f6c300d0fd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6c3008d280 tx=0x7f6c2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f97612378789700 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:55.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c2dffb700 1 -- 172.21.15.22:0/621353136 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2800e070 con 0x7f6c300b40c0 2023-07-06T01:19:55.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.663+0000 7f6c2dffb700 1 -- 172.21.15.22:0/621353136 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2800b910 con 0x7f6c300b40c0 2023-07-06T01:19:55.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.664+0000 7f6c2dffb700 1 -- 172.21.15.22:0/621353136 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2800bcc0 con 0x7f6c300b40c0 2023-07-06T01:19:55.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.664+0000 7f6c35491700 1 -- 172.21.15.22:0/621353136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c300b40c0 msgr2=0x7f6c300d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.664+0000 7f6c35491700 1 --2- 172.21.15.22:0/621353136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c300b40c0 0x7f6c300d0fd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6c3008d280 tx=0x7f6c2800b330 comp rx=0 tx=0).stop 2023-07-06T01:19:55.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.664+0000 7f6c35491700 1 -- 172.21.15.22:0/621353136 shutdown_connections 2023-07-06T01:19:55.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.664+0000 7f6c35491700 1 --2- 172.21.15.22:0/621353136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c300b40c0 0x7f6c300d0fd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.664+0000 7f6c35491700 1 --2- 172.21.15.22:0/621353136 >> v2:172.21.15.103:3300/0 conn(0x7f6c300b3b70 0x7f6c300d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.664+0000 7f6c35491700 1 -- 172.21.15.22:0/621353136 >> 172.21.15.22:0/621353136 conn(0x7f6c3009c2e0 msgr2=0x7f6c300d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:55.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.664+0000 7f6c35491700 1 -- 172.21.15.22:0/621353136 shutdown_connections 2023-07-06T01:19:55.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.664+0000 7f6c35491700 1 -- 172.21.15.22:0/621353136 wait complete. 2023-07-06T01:19:55.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.665+0000 7f6c35491700 1 Processor -- start 2023-07-06T01:19:55.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.665+0000 7f6c35491700 1 -- start start 2023-07-06T01:19:55.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.666+0000 7f6c35491700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c300b36f0 0x7f6c301584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:55.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.666+0000 7f6c35491700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c300b3b70 0x7f6c300c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:55.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.666+0000 7f6c35491700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c300b40c0 0x7f6c300cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:55.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.666+0000 7f6c35491700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c301589f0 con 0x7f6c300b36f0 2023-07-06T01:19:55.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.666+0000 7f6c35491700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300c8100 con 0x7f6c300b3b70 2023-07-06T01:19:55.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.666+0000 7f6c35491700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300c83e0 con 0x7f6c300b40c0 2023-07-06T01:19:55.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.666+0000 7f6c2effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c300b36f0 0x7f6c301584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:55.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.666+0000 7f6c2effd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c300b36f0 0x7f6c301584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46388/0 (socket says 172.21.15.22:46388) 2023-07-06T01:19:55.673 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.666+0000 7f6c2effd700 1 -- 172.21.15.22:0/2175322895 learned_addr learned my addr 172.21.15.22:0/2175322895 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:55.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.666+0000 7f6c2e7fc700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c300b3b70 0x7f6c300c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:55.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.666+0000 7f6c2f7fe700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c300b40c0 0x7f6c300cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:55.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.667+0000 7f6c2effd700 1 -- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c300b40c0 msgr2=0x7f6c300cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.667+0000 7f6c2effd700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c300b40c0 0x7f6c300cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.667+0000 7f6c2effd700 1 -- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c300b3b70 msgr2=0x7f6c300c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.667+0000 7f6c2effd700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c300b3b70 0x7f6c300c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.667+0000 7f6c2effd700 1 -- 172.21.15.22:0/2175322895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c28009b30 con 0x7f6c300b36f0 2023-07-06T01:19:55.674 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.667+0000 7f6c2e7fc700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c300b3b70 0x7f6c300c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:55.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.667+0000 7f6c2f7fe700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c300b40c0 0x7f6c300cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:55.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.667+0000 7f6c2effd700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c300b36f0 0x7f6c301584b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400c450 tx=0x7f6c2400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:55.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.667+0000 7f6c177fe700 1 -- 172.21.15.22:0/2175322895 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c240052d0 con 0x7f6c300b36f0 2023-07-06T01:19:55.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.667+0000 7f6c177fe700 1 -- 172.21.15.22:0/2175322895 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c24005430 con 0x7f6c300b36f0 2023-07-06T01:19:55.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.667+0000 7f6c35491700 1 -- 172.21.15.22:0/2175322895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c300c86c0 con 0x7f6c300b36f0 2023-07-06T01:19:55.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.667+0000 7f6c177fe700 1 -- 172.21.15.22:0/2175322895 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c240038c0 con 0x7f6c300b36f0 2023-07-06T01:19:55.675 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.667+0000 7f6c35491700 1 -- 172.21.15.22:0/2175322895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c300c8c80 con 0x7f6c300b36f0 2023-07-06T01:19:55.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.668+0000 7f6c35491700 1 -- 172.21.15.22:0/2175322895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c300b4840 con 0x7f6c300b36f0 2023-07-06T01:19:55.676 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.669+0000 7f6c177fe700 1 -- 172.21.15.22:0/2175322895 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c24003a20 con 0x7f6c300b36f0 2023-07-06T01:19:55.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.669+0000 7f6c177fe700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6c1c06ace0 0x7f6c1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:55.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.669+0000 7f6c177fe700 1 -- 172.21.15.22:0/2175322895 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c240888b0 con 0x7f6c300b36f0 2023-07-06T01:19:55.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.669+0000 7f6c2e7fc700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6c1c06ace0 0x7f6c1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:55.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.670+0000 7f6c2e7fc700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6c1c06ace0 0x7f6c1c06d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6c3015c8c0 tx=0x7f6c20008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:55.677 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.674+0000 7f6c177fe700 1 -- 172.21.15.22:0/2175322895 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6c24053960 con 0x7f6c300b36f0 2023-07-06T01:19:55.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.870+0000 7f6c35491700 1 -- 172.21.15.22:0/2175322895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6c300744b0 con 0x7f6c300b36f0 2023-07-06T01:19:55.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.870+0000 7f6c177fe700 1 -- 172.21.15.22:0/2175322895 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6c24056f80 con 0x7f6c300b36f0 2023-07-06T01:19:55.873 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:19:55.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.872+0000 7f6c35491700 1 -- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6c1c06ace0 msgr2=0x7f6c1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.872+0000 7f6c35491700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6c1c06ace0 0x7f6c1c06d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6c3015c8c0 tx=0x7f6c20008040 comp rx=0 tx=0).stop 2023-07-06T01:19:55.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.873+0000 7f6c35491700 1 -- 172.21.15.22:0/2175322895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c300b36f0 msgr2=0x7f6c301584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:55.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.873+0000 7f6c35491700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c300b36f0 0x7f6c301584b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400c450 tx=0x7f6c2400c810 comp rx=0 tx=0).stop 2023-07-06T01:19:55.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.873+0000 7f6c35491700 1 -- 172.21.15.22:0/2175322895 shutdown_connections 2023-07-06T01:19:55.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.873+0000 7f6c35491700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c300b40c0 0x7f6c300cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.873+0000 7f6c35491700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c300b36f0 0x7f6c301584b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.873+0000 7f6c35491700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6c1c06ace0 0x7f6c1c06d190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.873+0000 7f6c35491700 1 --2- 172.21.15.22:0/2175322895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c300b3b70 0x7f6c300c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:55.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.873+0000 7f6c35491700 1 -- 172.21.15.22:0/2175322895 >> 172.21.15.22:0/2175322895 conn(0x7f6c3009c2e0 msgr2=0x7f6c300cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:55.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.873+0000 7f6c35491700 1 -- 172.21.15.22:0/2175322895 shutdown_connections 2023-07-06T01:19:55.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:55.873+0000 7f6c35491700 1 -- 172.21.15.22:0/2175322895 wait complete. 2023-07-06T01:19:55.888 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:19:56.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.013+0000 7fe99f0b4700 1 Processor -- start 2023-07-06T01:19:56.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.013+0000 7fe99f0b4700 1 -- start start 2023-07-06T01:19:56.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe99f0b4700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe99815a6c0 0x7fe99815eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe99f0b4700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe998071980 0x7fe99806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe99f0b4700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99815f0b0 con 0x7fe99806a540 2023-07-06T01:19:56.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe99f0b4700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99806f260 con 0x7fe99815a6c0 2023-07-06T01:19:56.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe99f0b4700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99806a9e0 con 0x7fe998071980 2023-07-06T01:19:56.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe99ce50700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe99815a6c0 0x7fe99815eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe99d651700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe99806a540 0x7fe9980ca2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37142/0 (socket says 172.21.15.22:37142) 2023-07-06T01:19:56.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe99d651700 1 -- 172.21.15.22:0/1986117105 learned_addr learned my addr 172.21.15.22:0/1986117105 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:56.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe997fff700 1 --2- 172.21.15.22:0/1986117105 >> v2:172.21.15.103:3300/0 conn(0x7fe998071980 0x7fe99806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe99ce50700 1 -- 172.21.15.22:0/1986117105 >> v2:172.21.15.103:3300/0 conn(0x7fe998071980 msgr2=0x7fe99806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:56.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe99ce50700 1 --2- 172.21.15.22:0/1986117105 >> v2:172.21.15.103:3300/0 conn(0x7fe998071980 0x7fe99806a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe9977fe700 1 -- 172.21.15.22:0/1986117105 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548196885 0 0) 0x7fe990001490 con 0x7fe99806a540 2023-07-06T01:19:56.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe99ce50700 1 -- 172.21.15.22:0/1986117105 >> v1:172.21.15.22:6789/0 conn(0x7fe99806a540 legacy=0x7fe9980ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe99ce50700 1 -- 172.21.15.22:0/1986117105 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99815aa90 con 0x7fe99815a6c0 2023-07-06T01:19:56.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.014+0000 7fe99ce50700 1 --2- 172.21.15.22:0/1986117105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99815a6c0 0x7fe99815eb70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe9980cd700 tx=0x7fe98c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c946a0d0d2b88cb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:56.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.015+0000 7fe9977fe700 1 -- 172.21.15.22:0/1986117105 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c011830 con 0x7fe99815a6c0 2023-07-06T01:19:56.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.015+0000 7fe9977fe700 1 -- 172.21.15.22:0/1986117105 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c011990 con 0x7fe99815a6c0 2023-07-06T01:19:56.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.015+0000 7fe9977fe700 1 -- 172.21.15.22:0/1986117105 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c011d20 con 0x7fe99815a6c0 2023-07-06T01:19:56.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.015+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1986117105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99815a6c0 msgr2=0x7fe99815eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.015+0000 7fe99f0b4700 1 --2- 172.21.15.22:0/1986117105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99815a6c0 0x7fe99815eb70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe9980cd700 tx=0x7fe98c009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:56.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.015+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1986117105 shutdown_connections 2023-07-06T01:19:56.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.015+0000 7fe99f0b4700 1 --2- 172.21.15.22:0/1986117105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99815a6c0 0x7fe99815eb70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.015+0000 7fe99f0b4700 1 --2- 172.21.15.22:0/1986117105 >> v2:172.21.15.103:3300/0 conn(0x7fe998071980 0x7fe99806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.015+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1986117105 >> 172.21.15.22:0/1986117105 conn(0x7fe9980b1890 msgr2=0x7fe9980b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:56.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.015+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1986117105 shutdown_connections 2023-07-06T01:19:56.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.015+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1986117105 wait complete. 2023-07-06T01:19:56.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.016+0000 7fe99f0b4700 1 Processor -- start 2023-07-06T01:19:56.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.016+0000 7fe99f0b4700 1 -- start start 2023-07-06T01:19:56.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.017+0000 7fe99f0b4700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe99806a540 0x7fe998156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.017+0000 7fe99f0b4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99815a6c0 0x7fe9981598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.017+0000 7fe99f0b4700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe99815ae90 0x7fe9981540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.017+0000 7fe99f0b4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe998158b30 con 0x7fe99815a6c0 2023-07-06T01:19:56.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.017+0000 7fe99f0b4700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe998158e10 con 0x7fe99806a540 2023-07-06T01:19:56.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.017+0000 7fe99f0b4700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9981590f0 con 0x7fe99815ae90 2023-07-06T01:19:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.017+0000 7fe997fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99815a6c0 0x7fe9981598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.017+0000 7fe997fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99815a6c0 0x7fe9981598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46402/0 (socket says 172.21.15.22:46402) 2023-07-06T01:19:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.017+0000 7fe997fff700 1 -- 172.21.15.22:0/1136837041 learned_addr learned my addr 172.21.15.22:0/1136837041 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.017+0000 7fe99ce50700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe99806a540 0x7fe998156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.017+0000 7fe99d651700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe99815ae90 0x7fe9981540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.018+0000 7fe997fff700 1 -- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe99815ae90 msgr2=0x7fe9981540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.018+0000 7fe997fff700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe99815ae90 0x7fe9981540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.018+0000 7fe997fff700 1 -- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe99806a540 msgr2=0x7fe998156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.018+0000 7fe997fff700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe99806a540 0x7fe998156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.018+0000 7fe997fff700 1 -- 172.21.15.22:0/1136837041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c0094a0 con 0x7fe99815a6c0 2023-07-06T01:19:56.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.018+0000 7fe997fff700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99815a6c0 0x7fe9981598f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe98400e980 tx=0x7fe98400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:56.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.018+0000 7fe9957fa700 1 -- 172.21.15.22:0/1136837041 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe984013680 con 0x7fe99815a6c0 2023-07-06T01:19:56.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.018+0000 7fe9957fa700 1 -- 172.21.15.22:0/1136837041 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9840137e0 con 0x7fe99815a6c0 2023-07-06T01:19:56.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.018+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1136837041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9981593d0 con 0x7fe99815a6c0 2023-07-06T01:19:56.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.018+0000 7fe9957fa700 1 -- 172.21.15.22:0/1136837041 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe984013b50 con 0x7fe99815a6c0 2023-07-06T01:19:56.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.018+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1136837041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9981573f0 con 0x7fe99815a6c0 2023-07-06T01:19:56.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.019+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1136837041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9980744b0 con 0x7fe99815a6c0 2023-07-06T01:19:56.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.019+0000 7fe9957fa700 1 -- 172.21.15.22:0/1136837041 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe984013cb0 con 0x7fe99815a6c0 2023-07-06T01:19:56.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.020+0000 7fe9957fa700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe97c06ace0 0x7fe97c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.020+0000 7fe9957fa700 1 -- 172.21.15.22:0/1136837041 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9840893c0 con 0x7fe99815a6c0 2023-07-06T01:19:56.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.020+0000 7fe99ce50700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe97c06ace0 0x7fe97c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.021+0000 7fe99ce50700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe97c06ace0 0x7fe97c06d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0097c0 tx=0x7fe98c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:56.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.025+0000 7fe9957fa700 1 -- 172.21.15.22:0/1136837041 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe984054470 con 0x7fe99815a6c0 2023-07-06T01:19:56.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.221+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1136837041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe99806ee70 con 0x7fe99815a6c0 2023-07-06T01:19:56.224 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.221+0000 7fe9957fa700 1 -- 172.21.15.22:0/1136837041 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe984057a90 con 0x7fe99815a6c0 2023-07-06T01:19:56.224 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:19:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.223+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe97c06ace0 msgr2=0x7fe97c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.223+0000 7fe99f0b4700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe97c06ace0 0x7fe97c06d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0097c0 tx=0x7fe98c01c040 comp rx=0 tx=0).stop 2023-07-06T01:19:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.223+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1136837041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99815a6c0 msgr2=0x7fe9981598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.223+0000 7fe99f0b4700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99815a6c0 0x7fe9981598f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe98400e980 tx=0x7fe98400ec90 comp rx=0 tx=0).stop 2023-07-06T01:19:56.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.223+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1136837041 shutdown_connections 2023-07-06T01:19:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.223+0000 7fe99f0b4700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe99815ae90 0x7fe9981540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.223+0000 7fe99f0b4700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe97c06ace0 0x7fe97c06d190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.223+0000 7fe99f0b4700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe99806a540 0x7fe998156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.223+0000 7fe99f0b4700 1 --2- 172.21.15.22:0/1136837041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe99815a6c0 0x7fe9981598f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.223+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1136837041 >> 172.21.15.22:0/1136837041 conn(0x7fe9980b1890 msgr2=0x7fe9980b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.223+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1136837041 shutdown_connections 2023-07-06T01:19:56.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.223+0000 7fe99f0b4700 1 -- 172.21.15.22:0/1136837041 wait complete. 2023-07-06T01:19:56.238 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T01:19:56.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.363+0000 7fa56837a700 1 Processor -- start 2023-07-06T01:19:56.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.363+0000 7fa56837a700 1 -- start start 2023-07-06T01:19:56.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa56837a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5600cc320 0x7fa56006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa56837a700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5600af670 con 0x7fa5600af2a0 2023-07-06T01:19:56.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa56837a700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56006d030 con 0x7fa56006a710 2023-07-06T01:19:56.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa56837a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5600cc780 con 0x7fa5600cc320 2023-07-06T01:19:56.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa566116700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa5600af2a0 0x7fa56006a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37152/0 (socket says 172.21.15.22:37152) 2023-07-06T01:19:56.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa566116700 1 -- 172.21.15.22:0/1621312882 learned_addr learned my addr 172.21.15.22:0/1621312882 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:56.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa566917700 1 --2- 172.21.15.22:0/1621312882 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5600cc320 0x7fa56006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807509815 0 0) 0x7fa5600af670 con 0x7fa5600af2a0 2023-07-06T01:19:56.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa54c0035b0 con 0x7fa5600af2a0 2023-07-06T01:19:56.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4008999501 0 0) 0x7fa54c0035b0 con 0x7fa5600af2a0 2023-07-06T01:19:56.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5600cc320 msgr2=0x7fa56006caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:56.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 --2- 172.21.15.22:0/1621312882 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5600cc320 0x7fa56006caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 >> v1:172.21.15.103:6789/0 conn(0x7fa56006a710 legacy=0x7fa5600cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5600cd560 con 0x7fa5600af2a0 2023-07-06T01:19:56.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7fa55c001670 con 0x7fa5600af2a0 2023-07-06T01:19:56.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 >> v1:172.21.15.22:6789/0 conn(0x7fa5600af2a0 legacy=0x7fa56006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 --2- 172.21.15.22:0/1621312882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa54c003a30 0x7fa54c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 --2- 172.21.15.22:0/1621312882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa54c0043d0 0x7fa54c004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 --2- 172.21.15.22:0/1621312882 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5600cc320 0x7fa54c0050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5600cc780 con 0x7fa54c0043d0 2023-07-06T01:19:56.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c00ba10 con 0x7fa54c003a30 2023-07-06T01:19:56.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.364+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c00bba0 con 0x7fa5600cc320 2023-07-06T01:19:56.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.365+0000 7fa565915700 1 --2- 172.21.15.22:0/1621312882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa54c0043d0 0x7fa54c004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.365+0000 7fa566116700 1 --2- 172.21.15.22:0/1621312882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa54c003a30 0x7fa54c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.365+0000 7fa566917700 1 --2- 172.21.15.22:0/1621312882 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5600cc320 0x7fa54c0050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.365+0000 7fa565915700 1 -- 172.21.15.22:0/1621312882 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5600cc320 msgr2=0x7fa54c0050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:56.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.365+0000 7fa565915700 1 --2- 172.21.15.22:0/1621312882 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5600cc320 0x7fa54c0050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.365+0000 7fa565915700 1 -- 172.21.15.22:0/1621312882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa54c003a30 msgr2=0x7fa54c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:56.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.365+0000 7fa565915700 1 --2- 172.21.15.22:0/1621312882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa54c003a30 0x7fa54c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.365+0000 7fa565915700 1 -- 172.21.15.22:0/1621312882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c0035b0 con 0x7fa54c0043d0 2023-07-06T01:19:56.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.365+0000 7fa565915700 1 --2- 172.21.15.22:0/1621312882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa54c0043d0 0x7fa54c004890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa5600ca2a0 tx=0x7fa54c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601ac69dfb61acd9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:56.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.365+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550016070 con 0x7fa54c0043d0 2023-07-06T01:19:56.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.365+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa550006860 con 0x7fa54c0043d0 2023-07-06T01:19:56.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.365+0000 7fa565114700 1 -- 172.21.15.22:0/1621312882 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550001d80 con 0x7fa54c0043d0 2023-07-06T01:19:56.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.366+0000 7fa56837a700 1 -- 172.21.15.22:0/1621312882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa54c0043d0 msgr2=0x7fa54c004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.366+0000 7fa56837a700 1 --2- 172.21.15.22:0/1621312882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa54c0043d0 0x7fa54c004890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa5600ca2a0 tx=0x7fa54c005880 comp rx=0 tx=0).stop 2023-07-06T01:19:56.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.366+0000 7fa566917700 1 -- 172.21.15.22:0/1621312882 reap_dead start 2023-07-06T01:19:56.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.366+0000 7fa56837a700 1 -- 172.21.15.22:0/1621312882 shutdown_connections 2023-07-06T01:19:56.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.366+0000 7fa56837a700 1 -- 172.21.15.22:0/1621312882 >> 172.21.15.22:0/1621312882 conn(0x7fa5600b1890 msgr2=0x7fa5600b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:56.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.366+0000 7fa56837a700 1 -- 172.21.15.22:0/1621312882 shutdown_connections 2023-07-06T01:19:56.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.366+0000 7fa56837a700 1 -- 172.21.15.22:0/1621312882 wait complete. 2023-07-06T01:19:56.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.367+0000 7fa56837a700 1 Processor -- start 2023-07-06T01:19:56.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.367+0000 7fa56837a700 1 -- start start 2023-07-06T01:19:56.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa56837a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa54c0043d0 0x7fa5600b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa56837a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5600c7710 0x7fa5601f3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa56837a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5600b5370 0x7fa5600b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa56837a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5601f3690 con 0x7fa54c0043d0 2023-07-06T01:19:56.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa56837a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5600c7e60 con 0x7fa5600c7710 2023-07-06T01:19:56.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa56837a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5600c7b80 con 0x7fa5600b5370 2023-07-06T01:19:56.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa566116700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa54c0043d0 0x7fa5600b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa566116700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa54c0043d0 0x7fa5600b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46420/0 (socket says 172.21.15.22:46420) 2023-07-06T01:19:56.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa566116700 1 -- 172.21.15.22:0/2777406391 learned_addr learned my addr 172.21.15.22:0/2777406391 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:56.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa565915700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5600c7710 0x7fa5601f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa566917700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5600b5370 0x7fa5600b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa566116700 1 -- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5600b5370 msgr2=0x7fa5600b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa566116700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5600b5370 0x7fa5600b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa566116700 1 -- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5600c7710 msgr2=0x7fa5601f3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa566116700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5600c7710 0x7fa5601f3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa566116700 1 -- 172.21.15.22:0/2777406391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5601e53b0 con 0x7fa54c0043d0 2023-07-06T01:19:56.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa566116700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa54c0043d0 0x7fa5600b3a20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0011e0 tx=0x7fa55c007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:56.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.368+0000 7fa556ffd700 1 -- 172.21.15.22:0/2777406391 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c011580 con 0x7fa54c0043d0 2023-07-06T01:19:56.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.369+0000 7fa556ffd700 1 -- 172.21.15.22:0/2777406391 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c0116e0 con 0x7fa54c0043d0 2023-07-06T01:19:56.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.369+0000 7fa56837a700 1 -- 172.21.15.22:0/2777406391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5601e5690 con 0x7fa54c0043d0 2023-07-06T01:19:56.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.369+0000 7fa556ffd700 1 -- 172.21.15.22:0/2777406391 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c011a70 con 0x7fa54c0043d0 2023-07-06T01:19:56.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.369+0000 7fa56837a700 1 -- 172.21.15.22:0/2777406391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5600c6fe0 con 0x7fa54c0043d0 2023-07-06T01:19:56.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.370+0000 7fa56837a700 1 -- 172.21.15.22:0/2777406391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5600ae720 con 0x7fa54c0043d0 2023-07-06T01:19:56.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.370+0000 7fa556ffd700 1 -- 172.21.15.22:0/2777406391 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa55c011bd0 con 0x7fa54c0043d0 2023-07-06T01:19:56.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.370+0000 7fa556ffd700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa54406ace0 0x7fa54406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.370+0000 7fa556ffd700 1 -- 172.21.15.22:0/2777406391 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa55c089400 con 0x7fa54c0043d0 2023-07-06T01:19:56.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.371+0000 7fa565915700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa54406ace0 0x7fa54406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.371+0000 7fa565915700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa54406ace0 0x7fa54406d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa550014040 tx=0x7fa550009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:56.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.375+0000 7fa556ffd700 1 -- 172.21.15.22:0/2777406391 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa55c054570 con 0x7fa54c0043d0 2023-07-06T01:19:56.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.569+0000 7fa56837a700 1 -- 172.21.15.22:0/2777406391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5600744b0 con 0x7fa54c0043d0 2023-07-06T01:19:56.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.570+0000 7fa556ffd700 1 -- 172.21.15.22:0/2777406391 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa55c057b90 con 0x7fa54c0043d0 2023-07-06T01:19:56.573 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-07-06T01:19:56.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.571+0000 7fa56837a700 1 -- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa54406ace0 msgr2=0x7fa54406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.571+0000 7fa56837a700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa54406ace0 0x7fa54406d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa550014040 tx=0x7fa550009030 comp rx=0 tx=0).stop 2023-07-06T01:19:56.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.571+0000 7fa56837a700 1 -- 172.21.15.22:0/2777406391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa54c0043d0 msgr2=0x7fa5600b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.571+0000 7fa56837a700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa54c0043d0 0x7fa5600b3a20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0011e0 tx=0x7fa55c007570 comp rx=0 tx=0).stop 2023-07-06T01:19:56.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.571+0000 7fa56837a700 1 -- 172.21.15.22:0/2777406391 shutdown_connections 2023-07-06T01:19:56.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.571+0000 7fa56837a700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5600b5370 0x7fa5600b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.571+0000 7fa56837a700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa54c0043d0 0x7fa5600b3a20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.571+0000 7fa56837a700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa54406ace0 0x7fa54406d190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.571+0000 7fa56837a700 1 --2- 172.21.15.22:0/2777406391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5600c7710 0x7fa5601f3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.572+0000 7fa56837a700 1 -- 172.21.15.22:0/2777406391 >> 172.21.15.22:0/2777406391 conn(0x7fa5600b1890 msgr2=0x7fa5600cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:56.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.572+0000 7fa56837a700 1 -- 172.21.15.22:0/2777406391 shutdown_connections 2023-07-06T01:19:56.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.572+0000 7fa56837a700 1 -- 172.21.15.22:0/2777406391 wait complete. 2023-07-06T01:19:56.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T01:19:56.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.713+0000 7f0217774700 1 Processor -- start 2023-07-06T01:19:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.713+0000 7f0217774700 1 -- start start 2023-07-06T01:19:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0217774700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f02100af2a0 0x7f021006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0217774700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02100cc320 0x7f021006a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0217774700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02100af670 con 0x7f021006a000 2023-07-06T01:19:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0217774700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f021006a9e0 con 0x7f02100af2a0 2023-07-06T01:19:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0217774700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02100cd560 con 0x7f02100cc320 2023-07-06T01:19:56.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0215510700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f02100af2a0 0x7f021006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0214d0f700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f021006a000 0x7f02100cbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37172/0 (socket says 172.21.15.22:37172) 2023-07-06T01:19:56.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0214d0f700 1 -- 172.21.15.22:0/2862466448 learned_addr learned my addr 172.21.15.22:0/2862466448 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:56.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0215d11700 1 --2- 172.21.15.22:0/2862466448 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02100cc320 0x7f021006a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0215510700 1 -- 172.21.15.22:0/2862466448 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02100cc320 msgr2=0x7f021006a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:56.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0215510700 1 --2- 172.21.15.22:0/2862466448 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02100cc320 0x7f021006a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0215510700 1 -- 172.21.15.22:0/2862466448 >> v1:172.21.15.22:6789/0 conn(0x7f021006a000 legacy=0x7f02100cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0215510700 1 -- 172.21.15.22:0/2862466448 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021006d030 con 0x7f02100af2a0 2023-07-06T01:19:56.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0207fff700 1 -- 172.21.15.22:0/2862466448 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638301740 0 0) 0x7f02100af670 con 0x7f021006a000 2023-07-06T01:19:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0215d11700 1 --2- 172.21.15.22:0/2862466448 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02100cc320 0x7f021006a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0215510700 1 --2- 172.21.15.22:0/2862466448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02100af2a0 0x7f021006caf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f020c009840 tx=0x7f020c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3fbeb0330182372 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.714+0000 7f0207fff700 1 -- 172.21.15.22:0/2862466448 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c011830 con 0x7f02100af2a0 2023-07-06T01:19:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.715+0000 7f0207fff700 1 -- 172.21.15.22:0/2862466448 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020c011990 con 0x7f02100af2a0 2023-07-06T01:19:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.715+0000 7f0207fff700 1 -- 172.21.15.22:0/2862466448 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c011d20 con 0x7f02100af2a0 2023-07-06T01:19:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.715+0000 7f0217774700 1 -- 172.21.15.22:0/2862466448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02100af2a0 msgr2=0x7f021006caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.715+0000 7f0217774700 1 --2- 172.21.15.22:0/2862466448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02100af2a0 0x7f021006caf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f020c009840 tx=0x7f020c009b50 comp rx=0 tx=0).stop 2023-07-06T01:19:56.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.715+0000 7f0217774700 1 -- 172.21.15.22:0/2862466448 shutdown_connections 2023-07-06T01:19:56.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.715+0000 7f0217774700 1 --2- 172.21.15.22:0/2862466448 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02100cc320 0x7f021006a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.715+0000 7f0217774700 1 --2- 172.21.15.22:0/2862466448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02100af2a0 0x7f021006caf0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.715+0000 7f0217774700 1 -- 172.21.15.22:0/2862466448 >> 172.21.15.22:0/2862466448 conn(0x7f02100b1890 msgr2=0x7f02100b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:56.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.715+0000 7f0217774700 1 -- 172.21.15.22:0/2862466448 shutdown_connections 2023-07-06T01:19:56.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.715+0000 7f0217774700 1 -- 172.21.15.22:0/2862466448 wait complete. 2023-07-06T01:19:56.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.716+0000 7f0217774700 1 Processor -- start 2023-07-06T01:19:56.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.716+0000 7f0217774700 1 -- start start 2023-07-06T01:19:56.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.717+0000 7f0217774700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f021006a000 0x7f021015a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.717+0000 7f0217774700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02100af2a0 0x7f02101f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.717+0000 7f0217774700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02100cc320 0x7f02100b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.717+0000 7f0217774700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021015dc60 con 0x7f02100af2a0 2023-07-06T01:19:56.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.717+0000 7f0217774700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02101f3220 con 0x7f02100cc320 2023-07-06T01:19:56.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.717+0000 7f0217774700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021015a770 con 0x7f021006a000 2023-07-06T01:19:56.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.717+0000 7f0214d0f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02100af2a0 0x7f02101f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.717+0000 7f0214d0f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02100af2a0 0x7f02101f2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46436/0 (socket says 172.21.15.22:46436) 2023-07-06T01:19:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.717+0000 7f0214d0f700 1 -- 172.21.15.22:0/2581923209 learned_addr learned my addr 172.21.15.22:0/2581923209 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.717+0000 7f0215510700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f021006a000 0x7f021015a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.717+0000 7f0215d11700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02100cc320 0x7f02100b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.718+0000 7f0214d0f700 1 -- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f021006a000 msgr2=0x7f021015a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.718+0000 7f0214d0f700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f021006a000 0x7f021015a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.718+0000 7f0214d0f700 1 -- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02100cc320 msgr2=0x7f02100b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.718+0000 7f0214d0f700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02100cc320 0x7f02100b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.718+0000 7f0214d0f700 1 -- 172.21.15.22:0/2581923209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0094a0 con 0x7f02100af2a0 2023-07-06T01:19:56.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.718+0000 7f0214d0f700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02100af2a0 0x7f02101f2ce0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f02000011e0 tx=0x7f020000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:56.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.718+0000 7f0205ffb700 1 -- 172.21.15.22:0/2581923209 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200004c80 con 0x7f02100af2a0 2023-07-06T01:19:56.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.718+0000 7f0205ffb700 1 -- 172.21.15.22:0/2581923209 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0200004e00 con 0x7f02100af2a0 2023-07-06T01:19:56.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.718+0000 7f0217774700 1 -- 172.21.15.22:0/2581923209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021015aa50 con 0x7f02100af2a0 2023-07-06T01:19:56.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.718+0000 7f0205ffb700 1 -- 172.21.15.22:0/2581923209 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200007310 con 0x7f02100af2a0 2023-07-06T01:19:56.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.718+0000 7f0217774700 1 -- 172.21.15.22:0/2581923209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021015b010 con 0x7f02100af2a0 2023-07-06T01:19:56.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.719+0000 7f0217774700 1 -- 172.21.15.22:0/2581923209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021015ddd0 con 0x7f02100af2a0 2023-07-06T01:19:56.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.719+0000 7f0205ffb700 1 -- 172.21.15.22:0/2581923209 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0200007470 con 0x7f02100af2a0 2023-07-06T01:19:56.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.720+0000 7f0205ffb700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f01f406ace0 0x7f01f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:56.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.720+0000 7f0205ffb700 1 -- 172.21.15.22:0/2581923209 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0200089040 con 0x7f02100af2a0 2023-07-06T01:19:56.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.720+0000 7f0215510700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f01f406ace0 0x7f01f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:56.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.720+0000 7f0215510700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f01f406ace0 0x7f01f406d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f020c0090b0 tx=0x7f020c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:56.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.724+0000 7f0205ffb700 1 -- 172.21.15.22:0/2581923209 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0200054180 con 0x7f02100af2a0 2023-07-06T01:19:56.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.920+0000 7f0217774700 1 -- 172.21.15.22:0/2581923209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02100cf140 con 0x7f02100af2a0 2023-07-06T01:19:56.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.920+0000 7f0205ffb700 1 -- 172.21.15.22:0/2581923209 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02000577a0 con 0x7f02100af2a0 2023-07-06T01:19:56.923 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-07-06T01:19:56.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.922+0000 7f0217774700 1 -- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f01f406ace0 msgr2=0x7f01f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.922+0000 7f0217774700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f01f406ace0 0x7f01f406d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f020c0090b0 tx=0x7f020c008030 comp rx=0 tx=0).stop 2023-07-06T01:19:56.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.922+0000 7f0217774700 1 -- 172.21.15.22:0/2581923209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02100af2a0 msgr2=0x7f02101f2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:56.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.922+0000 7f0217774700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02100af2a0 0x7f02101f2ce0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f02000011e0 tx=0x7f020000ca30 comp rx=0 tx=0).stop 2023-07-06T01:19:56.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.922+0000 7f0217774700 1 -- 172.21.15.22:0/2581923209 shutdown_connections 2023-07-06T01:19:56.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.922+0000 7f0217774700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f01f406ace0 0x7f01f406d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.922+0000 7f0217774700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02100cc320 0x7f02100b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.922+0000 7f0217774700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f021006a000 0x7f021015a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.922+0000 7f0217774700 1 --2- 172.21.15.22:0/2581923209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02100af2a0 0x7f02101f2ce0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:56.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.922+0000 7f0217774700 1 -- 172.21.15.22:0/2581923209 >> 172.21.15.22:0/2581923209 conn(0x7f02100b1890 msgr2=0x7f02100ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:56.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.922+0000 7f0217774700 1 -- 172.21.15.22:0/2581923209 shutdown_connections 2023-07-06T01:19:56.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:56.922+0000 7f0217774700 1 -- 172.21.15.22:0/2581923209 wait complete. 2023-07-06T01:19:56.975 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-07-06T01:19:56.976 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2023-07-06T01:19:57.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.099+0000 7f71b9e8d700 1 Processor -- start 2023-07-06T01:19:57.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.100+0000 7f71b9e8d700 1 -- start start 2023-07-06T01:19:57.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.100+0000 7f71b9e8d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f71b415cb50 0x7f71b406a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.100+0000 7f71b9e8d700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f71b40cf260 0x7f71b4160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.100+0000 7f71b9e8d700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b40ca2b0 con 0x7f71b406a540 2023-07-06T01:19:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.100+0000 7f71b9e8d700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b40af440 con 0x7f71b415cb50 2023-07-06T01:19:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.100+0000 7f71b9e8d700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b406a970 con 0x7f71b40cf260 2023-07-06T01:19:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.100+0000 7f71b37fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f71b415cb50 0x7f71b406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:57.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.100+0000 7f71b2ffd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f71b406a540 0x7f71b40ceb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37186/0 (socket says 172.21.15.22:37186) 2023-07-06T01:19:57.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.100+0000 7f71b2ffd700 1 -- 172.21.15.22:0/1327805320 learned_addr learned my addr 172.21.15.22:0/1327805320 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:57.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.101+0000 7f71b3fff700 1 --2- 172.21.15.22:0/1327805320 >> v2:172.21.15.103:3300/0 conn(0x7f71b40cf260 0x7f71b4160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:57.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.101+0000 7f71b37fe700 1 -- 172.21.15.22:0/1327805320 >> v2:172.21.15.103:3300/0 conn(0x7f71b40cf260 msgr2=0x7f71b4160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:57.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.101+0000 7f71b37fe700 1 --2- 172.21.15.22:0/1327805320 >> v2:172.21.15.103:3300/0 conn(0x7f71b40cf260 0x7f71b4160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:57.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.101+0000 7f71b37fe700 1 -- 172.21.15.22:0/1327805320 >> v1:172.21.15.22:6789/0 conn(0x7f71b406a540 legacy=0x7f71b40ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:57.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.101+0000 7f71b27fc700 1 -- 172.21.15.22:0/1327805320 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995314383 0 0) 0x7f71b40ca2b0 con 0x7f71b406a540 2023-07-06T01:19:57.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.101+0000 7f71b37fe700 1 -- 172.21.15.22:0/1327805320 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b40cfc90 con 0x7f71b415cb50 2023-07-06T01:19:57.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.101+0000 7f71b3fff700 1 --2- 172.21.15.22:0/1327805320 >> v2:172.21.15.103:3300/0 conn(0x7f71b40cf260 0x7f71b4160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:57.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.101+0000 7f71b37fe700 1 --2- 172.21.15.22:0/1327805320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b415cb50 0x7f71b406a000 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f71b4071170 tx=0x7f71a4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40d2ae048c312de server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:57.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.101+0000 7f71b27fc700 1 -- 172.21.15.22:0/1327805320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4011830 con 0x7f71b415cb50 2023-07-06T01:19:57.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.101+0000 7f71b27fc700 1 -- 172.21.15.22:0/1327805320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a4011990 con 0x7f71b415cb50 2023-07-06T01:19:57.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.101+0000 7f71b27fc700 1 -- 172.21.15.22:0/1327805320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4011d20 con 0x7f71b415cb50 2023-07-06T01:19:57.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.102+0000 7f71b9e8d700 1 -- 172.21.15.22:0/1327805320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b415cb50 msgr2=0x7f71b406a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:57.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.102+0000 7f71b9e8d700 1 --2- 172.21.15.22:0/1327805320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b415cb50 0x7f71b406a000 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f71b4071170 tx=0x7f71a4009b20 comp rx=0 tx=0).stop 2023-07-06T01:19:57.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.102+0000 7f71b9e8d700 1 -- 172.21.15.22:0/1327805320 shutdown_connections 2023-07-06T01:19:57.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.102+0000 7f71b9e8d700 1 --2- 172.21.15.22:0/1327805320 >> v2:172.21.15.103:3300/0 conn(0x7f71b40cf260 0x7f71b4160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:57.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.102+0000 7f71b9e8d700 1 --2- 172.21.15.22:0/1327805320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b415cb50 0x7f71b406a000 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:57.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.102+0000 7f71b9e8d700 1 -- 172.21.15.22:0/1327805320 >> 172.21.15.22:0/1327805320 conn(0x7f71b406b110 msgr2=0x7f71b406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:57.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.102+0000 7f71b9e8d700 1 -- 172.21.15.22:0/1327805320 shutdown_connections 2023-07-06T01:19:57.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.102+0000 7f71b9e8d700 1 -- 172.21.15.22:0/1327805320 wait complete. 2023-07-06T01:19:57.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.103+0000 7f71b9e8d700 1 Processor -- start 2023-07-06T01:19:57.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.103+0000 7f71b9e8d700 1 -- start start 2023-07-06T01:19:57.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b9e8d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71b406a540 0x7f71b40b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:57.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b9e8d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b40cf260 0x7f71b40b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:57.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b9e8d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71b415cb50 0x7f71b41f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:57.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b9e8d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b415fce0 con 0x7f71b40cf260 2023-07-06T01:19:57.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b9e8d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b41f3660 con 0x7f71b415cb50 2023-07-06T01:19:57.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b9e8d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b40b4170 con 0x7f71b406a540 2023-07-06T01:19:57.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b40cf260 0x7f71b40b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:57.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b2ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b40cf260 0x7f71b40b5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46450/0 (socket says 172.21.15.22:46450) 2023-07-06T01:19:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b2ffd700 1 -- 172.21.15.22:0/208836917 learned_addr learned my addr 172.21.15.22:0/208836917 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b37fe700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71b406a540 0x7f71b40b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b3fff700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71b415cb50 0x7f71b41f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b2ffd700 1 -- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71b406a540 msgr2=0x7f71b40b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b2ffd700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71b406a540 0x7f71b40b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b2ffd700 1 -- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71b415cb50 msgr2=0x7f71b41f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:57.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b2ffd700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71b415cb50 0x7f71b41f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.104+0000 7f71b2ffd700 1 -- 172.21.15.22:0/208836917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a40094a0 con 0x7f71b40cf260 2023-07-06T01:19:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.105+0000 7f71b2ffd700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b40cf260 0x7f71b40b5210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71a80011e0 tx=0x7f71a800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.105+0000 7f719bfff700 1 -- 172.21.15.22:0/208836917 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8004c80 con 0x7f71b40cf260 2023-07-06T01:19:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.105+0000 7f719bfff700 1 -- 172.21.15.22:0/208836917 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a8004e00 con 0x7f71b40cf260 2023-07-06T01:19:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.105+0000 7f71b9e8d700 1 -- 172.21.15.22:0/208836917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b40b4450 con 0x7f71b40cf260 2023-07-06T01:19:57.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.105+0000 7f719bfff700 1 -- 172.21.15.22:0/208836917 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8007310 con 0x7f71b40cf260 2023-07-06T01:19:57.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.105+0000 7f71b9e8d700 1 -- 172.21.15.22:0/208836917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b40b4a10 con 0x7f71b40cf260 2023-07-06T01:19:57.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.106+0000 7f71b9e8d700 1 -- 172.21.15.22:0/208836917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b40ce4b0 con 0x7f71b40cf260 2023-07-06T01:19:57.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.106+0000 7f719bfff700 1 -- 172.21.15.22:0/208836917 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f71a8007470 con 0x7f71b40cf260 2023-07-06T01:19:57.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.107+0000 7f719bfff700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f71a006ace0 0x7f71a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:57.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.107+0000 7f719bfff700 1 -- 172.21.15.22:0/208836917 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f71a8089030 con 0x7f71b40cf260 2023-07-06T01:19:57.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.107+0000 7f71b37fe700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f71a006ace0 0x7f71a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:57.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.108+0000 7f71b37fe700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f71a006ace0 0x7f71a006d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f71a40090b0 tx=0x7f71a401c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:57.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.111+0000 7f719bfff700 1 -- 172.21.15.22:0/208836917 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f71a8054170 con 0x7f71b40cf260 2023-07-06T01:19:57.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:57.392+0000 7f71b9e8d700 1 -- 172.21.15.22:0/208836917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f71b40744b0 con 0x7f71b40cf260 2023-07-06T01:19:58.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.236+0000 7f719bfff700 1 -- 172.21.15.22:0/208836917 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v50) v1 ==== 151+0+0 (secure 0 0 0) 0x7f71a8057790 con 0x7f71b40cf260 2023-07-06T01:19:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.239+0000 7f71b9e8d700 1 -- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f71a006ace0 msgr2=0x7f71a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.239+0000 7f71b9e8d700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f71a006ace0 0x7f71a006d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f71a40090b0 tx=0x7f71a401c040 comp rx=0 tx=0).stop 2023-07-06T01:19:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.239+0000 7f71b9e8d700 1 -- 172.21.15.22:0/208836917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b40cf260 msgr2=0x7f71b40b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.239+0000 7f71b9e8d700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b40cf260 0x7f71b40b5210 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71a80011e0 tx=0x7f71a800ca30 comp rx=0 tx=0).stop 2023-07-06T01:19:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.239+0000 7f71b9e8d700 1 -- 172.21.15.22:0/208836917 shutdown_connections 2023-07-06T01:19:58.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.239+0000 7f71b9e8d700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f71a006ace0 0x7f71a006d190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.239+0000 7f71b9e8d700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71b415cb50 0x7f71b41f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.239+0000 7f71b9e8d700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71b406a540 0x7f71b40b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.239+0000 7f71b9e8d700 1 --2- 172.21.15.22:0/208836917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b40cf260 0x7f71b40b5210 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.239+0000 7f71b9e8d700 1 -- 172.21.15.22:0/208836917 >> 172.21.15.22:0/208836917 conn(0x7f71b406b110 msgr2=0x7f71b406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:58.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.239+0000 7f71b9e8d700 1 -- 172.21.15.22:0/208836917 shutdown_connections 2023-07-06T01:19:58.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.239+0000 7f71b9e8d700 1 -- 172.21.15.22:0/208836917 wait complete. 2023-07-06T01:19:58.257 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T01:19:58.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.389+0000 7fc8e4bb0700 1 Processor -- start 2023-07-06T01:19:58.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.389+0000 7fc8e4bb0700 1 -- start start 2023-07-06T01:19:58.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8e4bb0700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fc8e00ccd70 0x7fc8e015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:58.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8e4bb0700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e00cf230 con 0x7fc8e006d040 2023-07-06T01:19:58.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8e4bb0700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e00ce620 con 0x7fc8e0159360 2023-07-06T01:19:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8e4bb0700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e015df20 con 0x7fc8e00ccd70 2023-07-06T01:19:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8ded9d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc8e006d040 0x7fc8e006d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37204/0 (socket says 172.21.15.22:37204) 2023-07-06T01:19:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8ded9d700 1 -- 172.21.15.22:0/1916673459 learned_addr learned my addr 172.21.15.22:0/1916673459 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:19:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8ddd9b700 1 --2- 172.21.15.22:0/1916673459 >> v2:172.21.15.103:3300/0 conn(0x7fc8e00ccd70 0x7fc8e015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312522954 0 0) 0x7fc8e00cf230 con 0x7fc8e006d040 2023-07-06T01:19:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8c80035b0 con 0x7fc8e006d040 2023-07-06T01:19:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1026105110 0 0) 0x7fc8c80035b0 con 0x7fc8e006d040 2023-07-06T01:19:58.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 >> v2:172.21.15.103:3300/0 conn(0x7fc8e00ccd70 msgr2=0x7fc8e015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:58.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 --2- 172.21.15.22:0/1916673459 >> v2:172.21.15.103:3300/0 conn(0x7fc8e00ccd70 0x7fc8e015d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 >> v1:172.21.15.103:6789/0 conn(0x7fc8e0159360 legacy=0x7fc8e00cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:58.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e00ce900 con 0x7fc8e006d040 2023-07-06T01:19:58.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7fc8d80019f0 con 0x7fc8e006d040 2023-07-06T01:19:58.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 >> v1:172.21.15.22:6789/0 conn(0x7fc8e006d040 legacy=0x7fc8e006d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:58.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 --2- 172.21.15.22:0/1916673459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8c8003a00 0x7fc8c8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:58.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 --2- 172.21.15.22:0/1916673459 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8c8004850 0x7fc8c8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:58.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 --2- 172.21.15.22:0/1916673459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c8005200 0x7fc8c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:58.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e015df20 con 0x7fc8c8005200 2023-07-06T01:19:58.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c800c080 con 0x7fc8c8003a00 2023-07-06T01:19:58.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.390+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c800c240 con 0x7fc8c8004850 2023-07-06T01:19:58.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8ded9d700 1 --2- 172.21.15.22:0/1916673459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c8005200 0x7fc8c800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:58.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8de59c700 1 --2- 172.21.15.22:0/1916673459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8c8003a00 0x7fc8c8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:58.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8ddd9b700 1 --2- 172.21.15.22:0/1916673459 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8c8004850 0x7fc8c8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:58.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8ded9d700 1 -- 172.21.15.22:0/1916673459 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8c8004850 msgr2=0x7fc8c8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:58.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8ded9d700 1 --2- 172.21.15.22:0/1916673459 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8c8004850 0x7fc8c8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8ded9d700 1 -- 172.21.15.22:0/1916673459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8c8003a00 msgr2=0x7fc8c8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:58.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8ded9d700 1 --2- 172.21.15.22:0/1916673459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8c8003a00 0x7fc8c8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8ded9d700 1 -- 172.21.15.22:0/1916673459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c80035b0 con 0x7fc8c8005200 2023-07-06T01:19:58.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8ddd9b700 1 --2- 172.21.15.22:0/1916673459 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8c8004850 0x7fc8c8004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:58.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8de59c700 1 --2- 172.21.15.22:0/1916673459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8c8003a00 0x7fc8c8003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:58.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8ded9d700 1 -- 172.21.15.22:0/1916673459 reap_dead start 2023-07-06T01:19:58.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8ded9d700 1 --2- 172.21.15.22:0/1916673459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c8005200 0x7fc8c800ba10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800a830 tx=0x7fc8c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80855a4f6400140 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800a2f0 con 0x7fc8c8005200 2023-07-06T01:19:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d8012600 con 0x7fc8c8005200 2023-07-06T01:19:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.391+0000 7fc8dd59a700 1 -- 172.21.15.22:0/1916673459 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d80129b0 con 0x7fc8c8005200 2023-07-06T01:19:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.392+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/1916673459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c8005200 msgr2=0x7fc8c800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.392+0000 7fc8e4bb0700 1 --2- 172.21.15.22:0/1916673459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c8005200 0x7fc8c800ba10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800a830 tx=0x7fc8c8005880 comp rx=0 tx=0).stop 2023-07-06T01:19:58.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.392+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/1916673459 shutdown_connections 2023-07-06T01:19:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.392+0000 7fc8e4bb0700 1 --2- 172.21.15.22:0/1916673459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c8005200 0x7fc8c800ba10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.392+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/1916673459 >> 172.21.15.22:0/1916673459 conn(0x7fc8e00b1890 msgr2=0x7fc8e00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.392+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/1916673459 shutdown_connections 2023-07-06T01:19:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.392+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/1916673459 wait complete. 2023-07-06T01:19:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.393+0000 7fc8e4bb0700 1 Processor -- start 2023-07-06T01:19:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.394+0000 7fc8e4bb0700 1 -- start start 2023-07-06T01:19:58.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.394+0000 7fc8e4bb0700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c8005200 0x7fc8e014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.394+0000 7fc8e4bb0700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e00c6d70 0x7fc8e015b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.394+0000 7fc8e4bb0700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8e015bcc0 0x7fc8e00b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8e4bb0700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e015b0e0 con 0x7fc8c8005200 2023-07-06T01:19:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8e4bb0700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e015a520 con 0x7fc8e00c6d70 2023-07-06T01:19:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8e4bb0700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e014d420 con 0x7fc8e015bcc0 2023-07-06T01:19:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8de59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c8005200 0x7fc8e014cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:58.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8de59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c8005200 0x7fc8e014cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46468/0 (socket says 172.21.15.22:46468) 2023-07-06T01:19:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8de59c700 1 -- 172.21.15.22:0/469665987 learned_addr learned my addr 172.21.15.22:0/469665987 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8ddd9b700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e00c6d70 0x7fc8e015b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8ded9d700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8e015bcc0 0x7fc8e00b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:58.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8de59c700 1 -- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8e015bcc0 msgr2=0x7fc8e00b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8de59c700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8e015bcc0 0x7fc8e00b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8de59c700 1 -- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e00c6d70 msgr2=0x7fc8e015b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:58.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8de59c700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e00c6d70 0x7fc8e015b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8de59c700 1 -- 172.21.15.22:0/469665987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e00c7510 con 0x7fc8c8005200 2023-07-06T01:19:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8ded9d700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8e015bcc0 0x7fc8e00b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8ddd9b700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e00c6d70 0x7fc8e015b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:58.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8de59c700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c8005200 0x7fc8e014cee0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00107c0 tx=0x7fc8d0010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8ceffd700 1 -- 172.21.15.22:0/469665987 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00054e0 con 0x7fc8c8005200 2023-07-06T01:19:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8ceffd700 1 -- 172.21.15.22:0/469665987 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d0003680 con 0x7fc8c8005200 2023-07-06T01:19:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/469665987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e015a740 con 0x7fc8c8005200 2023-07-06T01:19:58.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8ceffd700 1 -- 172.21.15.22:0/469665987 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0003a10 con 0x7fc8c8005200 2023-07-06T01:19:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.395+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/469665987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e015ad00 con 0x7fc8c8005200 2023-07-06T01:19:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.396+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/469665987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e00744b0 con 0x7fc8c8005200 2023-07-06T01:19:58.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.397+0000 7fc8ceffd700 1 -- 172.21.15.22:0/469665987 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc8d0003b70 con 0x7fc8c8005200 2023-07-06T01:19:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.397+0000 7fc8ceffd700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc8c006ace0 0x7fc8c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.397+0000 7fc8ceffd700 1 -- 172.21.15.22:0/469665987 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc8d008b3e0 con 0x7fc8c8005200 2023-07-06T01:19:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.397+0000 7fc8ddd9b700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc8c006ace0 0x7fc8c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:58.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.398+0000 7fc8ddd9b700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc8c006ace0 0x7fc8c006d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc8e006f420 tx=0x7fc8d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:58.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.401+0000 7fc8ceffd700 1 -- 172.21.15.22:0/469665987 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc8d0056520 con 0x7fc8c8005200 2023-07-06T01:19:58.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.603+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/469665987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc8e015c320 con 0x7fc8c8005200 2023-07-06T01:19:58.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.606+0000 7fc8ceffd700 1 -- 172.21.15.22:0/469665987 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc8d0059b40 con 0x7fc8c8005200 2023-07-06T01:19:58.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.608+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc8c006ace0 msgr2=0x7fc8c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:58.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.608+0000 7fc8e4bb0700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc8c006ace0 0x7fc8c006d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc8e006f420 tx=0x7fc8d4008040 comp rx=0 tx=0).stop 2023-07-06T01:19:58.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.608+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/469665987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c8005200 msgr2=0x7fc8e014cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:58.611 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.608+0000 7fc8e4bb0700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c8005200 0x7fc8e014cee0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00107c0 tx=0x7fc8d0010b80 comp rx=0 tx=0).stop 2023-07-06T01:19:58.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.608+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/469665987 shutdown_connections 2023-07-06T01:19:58.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.608+0000 7fc8e4bb0700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc8e015bcc0 0x7fc8e00b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.608+0000 7fc8e4bb0700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc8c8005200 0x7fc8e014cee0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.608+0000 7fc8e4bb0700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc8c006ace0 0x7fc8c006d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.608+0000 7fc8e4bb0700 1 --2- 172.21.15.22:0/469665987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8e00c6d70 0x7fc8e015b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:58.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.608+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/469665987 >> 172.21.15.22:0/469665987 conn(0x7fc8e00b1890 msgr2=0x7fc8e0156d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:58.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.608+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/469665987 shutdown_connections 2023-07-06T01:19:58.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:58.608+0000 7fc8e4bb0700 1 -- 172.21.15.22:0/469665987 wait complete. 2023-07-06T01:19:58.623 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T01:19:58.623 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T01:19:58.662 DEBUG:teuthology.orchestra.run.smithi103:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T01:19:58.708 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T01:19:58.709 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:58.709 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:19:58.727 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:58.727 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:19:58.783 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:19:58.784 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:19:58.841 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:19:59.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.003+0000 7f00ecab2700 1 Processor -- start 2023-07-06T01:19:59.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.003+0000 7f00ecab2700 1 -- start start 2023-07-06T01:19:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00ecab2700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f00e815a710 0x7f00e80ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00ecab2700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00e80aecb0 0x7f00e80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00ecab2700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e80af270 con 0x7f00e80ceda0 2023-07-06T01:19:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00ecab2700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e80cf220 con 0x7f00e815a710 2023-07-06T01:19:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00ecab2700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e80af550 con 0x7f00e80aecb0 2023-07-06T01:19:59.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00e659c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f00e815a710 0x7f00e80ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00e659c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f00e815a710 0x7f00e80ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46476/0 (socket says 172.21.15.22:46476) 2023-07-06T01:19:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00e659c700 1 -- 172.21.15.22:0/2470972480 learned_addr learned my addr 172.21.15.22:0/2470972480 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00e5d9b700 1 --2- 172.21.15.22:0/2470972480 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00e80aecb0 0x7f00e80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00e659c700 1 -- 172.21.15.22:0/2470972480 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00e80aecb0 msgr2=0x7f00e80ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00e659c700 1 --2- 172.21.15.22:0/2470972480 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00e80aecb0 0x7f00e80ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00e659c700 1 -- 172.21.15.22:0/2470972480 >> v1:172.21.15.103:6789/0 conn(0x7f00e80ceda0 legacy=0x7f00e815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00e659c700 1 -- 172.21.15.22:0/2470972480 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e80af830 con 0x7f00e815a710 2023-07-06T01:19:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00e5d9b700 1 --2- 172.21.15.22:0/2470972480 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00e80aecb0 0x7f00e80ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00e659c700 1 --2- 172.21.15.22:0/2470972480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e815a710 0x7f00e80ae770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0098f0 tx=0x7f00dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee91f7b7a81169f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00e559a700 1 -- 172.21.15.22:0/2470972480 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc004230 con 0x7f00e815a710 2023-07-06T01:19:59.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00e559a700 1 -- 172.21.15.22:0/2470972480 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00dc004390 con 0x7f00e815a710 2023-07-06T01:19:59.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.004+0000 7f00e559a700 1 -- 172.21.15.22:0/2470972480 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0036a0 con 0x7f00e815a710 2023-07-06T01:19:59.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.005+0000 7f00ecab2700 1 -- 172.21.15.22:0/2470972480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e815a710 msgr2=0x7f00e80ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.005+0000 7f00ecab2700 1 --2- 172.21.15.22:0/2470972480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e815a710 0x7f00e80ae770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0098f0 tx=0x7f00dc009c00 comp rx=0 tx=0).stop 2023-07-06T01:19:59.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.005+0000 7f00ecab2700 1 -- 172.21.15.22:0/2470972480 shutdown_connections 2023-07-06T01:19:59.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.005+0000 7f00ecab2700 1 --2- 172.21.15.22:0/2470972480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e815a710 0x7f00e80ae770 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.005+0000 7f00ecab2700 1 --2- 172.21.15.22:0/2470972480 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00e80aecb0 0x7f00e80ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.005+0000 7f00ecab2700 1 -- 172.21.15.22:0/2470972480 >> 172.21.15.22:0/2470972480 conn(0x7f00e80999f0 msgr2=0x7f00e8099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:59.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.005+0000 7f00ecab2700 1 -- 172.21.15.22:0/2470972480 shutdown_connections 2023-07-06T01:19:59.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.005+0000 7f00ecab2700 1 -- 172.21.15.22:0/2470972480 wait complete. 2023-07-06T01:19:59.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.006+0000 7f00ecab2700 1 Processor -- start 2023-07-06T01:19:59.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.006+0000 7f00ecab2700 1 -- start start 2023-07-06T01:19:59.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.007+0000 7f00ecab2700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00e80aecb0 0x7f00e80b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.007+0000 7f00ecab2700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00e80ceda0 0x7f00e80b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.007+0000 7f00ecab2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e815a710 0x7f00e80cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.007+0000 7f00ecab2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e81f4210 con 0x7f00e815a710 2023-07-06T01:19:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.007+0000 7f00ecab2700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e81efbe0 con 0x7f00e80aecb0 2023-07-06T01:19:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.007+0000 7f00ecab2700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e81f3690 con 0x7f00e80ceda0 2023-07-06T01:19:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.007+0000 7f00e6d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e815a710 0x7f00e80cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.017 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.007+0000 7f00e6d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e815a710 0x7f00e80cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46484/0 (socket says 172.21.15.22:46484) 2023-07-06T01:19:59.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.007+0000 7f00e6d9d700 1 -- 172.21.15.22:0/1328528597 learned_addr learned my addr 172.21.15.22:0/1328528597 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:59.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.007+0000 7f00e659c700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00e80aecb0 0x7f00e80b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.007+0000 7f00e5d9b700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00e80ceda0 0x7f00e80b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.008+0000 7f00e6d9d700 1 -- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00e80ceda0 msgr2=0x7f00e80b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.018 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.008+0000 7f00e6d9d700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00e80ceda0 0x7f00e80b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.008+0000 7f00e6d9d700 1 -- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00e80aecb0 msgr2=0x7f00e80b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.008+0000 7f00e6d9d700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00e80aecb0 0x7f00e80b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.008+0000 7f00e6d9d700 1 -- 172.21.15.22:0/1328528597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc009550 con 0x7f00e815a710 2023-07-06T01:19:59.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.008+0000 7f00e5d9b700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00e80ceda0 0x7f00e80b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:19:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.008+0000 7f00e6d9d700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e815a710 0x7f00e80cd880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f00e000cc10 tx=0x7f00e000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.008+0000 7f00d6ffd700 1 -- 172.21.15.22:0/1328528597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e000ab10 con 0x7f00e815a710 2023-07-06T01:19:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.008+0000 7f00d6ffd700 1 -- 172.21.15.22:0/1328528597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e000ac70 con 0x7f00e815a710 2023-07-06T01:19:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.008+0000 7f00ecab2700 1 -- 172.21.15.22:0/1328528597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e81f3970 con 0x7f00e815a710 2023-07-06T01:19:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.008+0000 7f00d6ffd700 1 -- 172.21.15.22:0/1328528597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0012420 con 0x7f00e815a710 2023-07-06T01:19:59.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.008+0000 7f00ecab2700 1 -- 172.21.15.22:0/1328528597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e81f3f30 con 0x7f00e815a710 2023-07-06T01:19:59.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.009+0000 7f00ecab2700 1 -- 172.21.15.22:0/1328528597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e80b4ac0 con 0x7f00e815a710 2023-07-06T01:19:59.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.009+0000 7f00d6ffd700 1 -- 172.21.15.22:0/1328528597 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f00e0012580 con 0x7f00e815a710 2023-07-06T01:19:59.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.010+0000 7f00d6ffd700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f00c806ace0 0x7f00c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.010+0000 7f00d6ffd700 1 -- 172.21.15.22:0/1328528597 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f00e00897c0 con 0x7f00e815a710 2023-07-06T01:19:59.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.010+0000 7f00e659c700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f00c806ace0 0x7f00c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.010+0000 7f00e659c700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f00c806ace0 0x7f00c806d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009840 tx=0x7f00dc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:59.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.014+0000 7f00d6ffd700 1 -- 172.21.15.22:0/1328528597 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f00e0054870 con 0x7f00e815a710 2023-07-06T01:19:59.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.211+0000 7f00ecab2700 1 -- 172.21.15.22:0/1328528597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00e80744b0 con 0x7f00e815a710 2023-07-06T01:19:59.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.213+0000 7f00d6ffd700 1 -- 172.21.15.22:0/1328528597 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7f00e0057e90 con 0x7f00e815a710 2023-07-06T01:19:59.216 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:19:59.216 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:19:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.215+0000 7f00ecab2700 1 -- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f00c806ace0 msgr2=0x7f00c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.215+0000 7f00ecab2700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f00c806ace0 0x7f00c806d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009840 tx=0x7f00dc008030 comp rx=0 tx=0).stop 2023-07-06T01:19:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.215+0000 7f00ecab2700 1 -- 172.21.15.22:0/1328528597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e815a710 msgr2=0x7f00e80cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.215+0000 7f00ecab2700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e815a710 0x7f00e80cd880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f00e000cc10 tx=0x7f00e000cf20 comp rx=0 tx=0).stop 2023-07-06T01:19:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.215+0000 7f00ecab2700 1 -- 172.21.15.22:0/1328528597 shutdown_connections 2023-07-06T01:19:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.215+0000 7f00ecab2700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00e815a710 0x7f00e80cd880 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.215+0000 7f00ecab2700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f00c806ace0 0x7f00c806d190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.215+0000 7f00ecab2700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00e80aecb0 0x7f00e80b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.215+0000 7f00ecab2700 1 --2- 172.21.15.22:0/1328528597 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00e80ceda0 0x7f00e80b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.215+0000 7f00ecab2700 1 -- 172.21.15.22:0/1328528597 >> 172.21.15.22:0/1328528597 conn(0x7f00e80999f0 msgr2=0x7f00e80cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.215+0000 7f00ecab2700 1 -- 172.21.15.22:0/1328528597 shutdown_connections 2023-07-06T01:19:59.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.215+0000 7f00ecab2700 1 -- 172.21.15.22:0/1328528597 wait complete. 2023-07-06T01:19:59.230 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T01:19:22.873255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T01:19:24.019088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:19:59.232 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:19:59.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.358+0000 7fa7cade8700 1 Processor -- start 2023-07-06T01:19:59.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.358+0000 7fa7cade8700 1 -- start start 2023-07-06T01:19:59.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.359+0000 7fa7cade8700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7c40ce860 0x7fa7c40cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.359+0000 7fa7cade8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa7c40cbae0 0x7fa7c40cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.359+0000 7fa7cade8700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c40af270 con 0x7fa7c415a710 2023-07-06T01:19:59.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.359+0000 7fa7cade8700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c40cf210 con 0x7fa7c40cbae0 2023-07-06T01:19:59.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.359+0000 7fa7cade8700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40af550 con 0x7fa7c40ce860 2023-07-06T01:19:59.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.359+0000 7fa7c9385700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa7c40cbae0 0x7fa7c40cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.359+0000 7fa7c9385700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa7c40cbae0 0x7fa7c40cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46494/0 (socket says 172.21.15.22:46494) 2023-07-06T01:19:59.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.359+0000 7fa7c9385700 1 -- 172.21.15.22:0/3668669707 learned_addr learned my addr 172.21.15.22:0/3668669707 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:59.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.359+0000 7fa7c3fff700 1 --2- 172.21.15.22:0/3668669707 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7c40ce860 0x7fa7c40cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7c9385700 1 -- 172.21.15.22:0/3668669707 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7c40ce860 msgr2=0x7fa7c40cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:59.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7c9385700 1 --2- 172.21.15.22:0/3668669707 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7c40ce860 0x7fa7c40cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7c9385700 1 -- 172.21.15.22:0/3668669707 >> v1:172.21.15.103:6789/0 conn(0x7fa7c415a710 legacy=0x7fa7c40ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:59.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7c9385700 1 -- 172.21.15.22:0/3668669707 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c40af830 con 0x7fa7c40cbae0 2023-07-06T01:19:59.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7c9385700 1 --2- 172.21.15.22:0/3668669707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7c40cbae0 0x7fa7c40cbf60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009e50 tx=0x7fa7bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdc1a38f90b1ebb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:59.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7c37fe700 1 -- 172.21.15.22:0/3668669707 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc00e070 con 0x7fa7c40cbae0 2023-07-06T01:19:59.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7c37fe700 1 -- 172.21.15.22:0/3668669707 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc003a20 con 0x7fa7c40cbae0 2023-07-06T01:19:59.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7c37fe700 1 -- 172.21.15.22:0/3668669707 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc003dd0 con 0x7fa7c40cbae0 2023-07-06T01:19:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7cade8700 1 -- 172.21.15.22:0/3668669707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7c40cbae0 msgr2=0x7fa7c40cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7cade8700 1 --2- 172.21.15.22:0/3668669707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7c40cbae0 0x7fa7c40cbf60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009e50 tx=0x7fa7bc00b330 comp rx=0 tx=0).stop 2023-07-06T01:19:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7cade8700 1 -- 172.21.15.22:0/3668669707 shutdown_connections 2023-07-06T01:19:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7cade8700 1 --2- 172.21.15.22:0/3668669707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7c40cbae0 0x7fa7c40cbf60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7cade8700 1 --2- 172.21.15.22:0/3668669707 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7c40ce860 0x7fa7c40cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7cade8700 1 -- 172.21.15.22:0/3668669707 >> 172.21.15.22:0/3668669707 conn(0x7fa7c40999f0 msgr2=0x7fa7c4099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7cade8700 1 -- 172.21.15.22:0/3668669707 shutdown_connections 2023-07-06T01:19:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.360+0000 7fa7cade8700 1 -- 172.21.15.22:0/3668669707 wait complete. 2023-07-06T01:19:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.362+0000 7fa7cade8700 1 Processor -- start 2023-07-06T01:19:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.362+0000 7fa7cade8700 1 -- start start 2023-07-06T01:19:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.363+0000 7fa7cade8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7c40cbae0 0x7fa7c41ef780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.363+0000 7fa7cade8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7c40ce860 0x7fa7c40c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.363+0000 7fa7cade8700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7c415a710 0x7fa7c40c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.363+0000 7fa7cade8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40c9ad0 con 0x7fa7c40ce860 2023-07-06T01:19:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.363+0000 7fa7cade8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40c8e50 con 0x7fa7c40cbae0 2023-07-06T01:19:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.363+0000 7fa7cade8700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40c9130 con 0x7fa7c415a710 2023-07-06T01:19:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.363+0000 7fa7c3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7c40ce860 0x7fa7c40c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.363+0000 7fa7c3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7c40ce860 0x7fa7c40c47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46498/0 (socket says 172.21.15.22:46498) 2023-07-06T01:19:59.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.363+0000 7fa7c3fff700 1 -- 172.21.15.22:0/4166226882 learned_addr learned my addr 172.21.15.22:0/4166226882 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:59.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7c8b84700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7c40cbae0 0x7fa7c41ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7c9385700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7c415a710 0x7fa7c40c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7c3fff700 1 -- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7c415a710 msgr2=0x7fa7c40c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7c3fff700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7c415a710 0x7fa7c40c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7c3fff700 1 -- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7c40cbae0 msgr2=0x7fa7c41ef780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7c3fff700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7c40cbae0 0x7fa7c41ef780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7c3fff700 1 -- 172.21.15.22:0/4166226882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc009b30 con 0x7fa7c40ce860 2023-07-06T01:19:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7c8b84700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7c40cbae0 0x7fa7c41ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7c9385700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7c415a710 0x7fa7c40c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7c3fff700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7c40ce860 0x7fa7c40c47c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000e810 tx=0x7fa7b000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7c17fa700 1 -- 172.21.15.22:0/4166226882 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0013610 con 0x7fa7c40ce860 2023-07-06T01:19:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7c17fa700 1 -- 172.21.15.22:0/4166226882 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b0013770 con 0x7fa7c40ce860 2023-07-06T01:19:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7cade8700 1 -- 172.21.15.22:0/4166226882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c40c9410 con 0x7fa7c40ce860 2023-07-06T01:19:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7c17fa700 1 -- 172.21.15.22:0/4166226882 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0013b20 con 0x7fa7c40ce860 2023-07-06T01:19:59.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.364+0000 7fa7cade8700 1 -- 172.21.15.22:0/4166226882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c41efcc0 con 0x7fa7c40ce860 2023-07-06T01:19:59.372 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.365+0000 7fa7cade8700 1 -- 172.21.15.22:0/4166226882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c40744b0 con 0x7fa7c40ce860 2023-07-06T01:19:59.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.365+0000 7fa7c17fa700 1 -- 172.21.15.22:0/4166226882 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7b0013c80 con 0x7fa7c40ce860 2023-07-06T01:19:59.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.366+0000 7fa7c17fa700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa7a806ace0 0x7fa7a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.366+0000 7fa7c17fa700 1 -- 172.21.15.22:0/4166226882 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7b0089100 con 0x7fa7c40ce860 2023-07-06T01:19:59.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.366+0000 7fa7c8b84700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa7a806ace0 0x7fa7a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.367+0000 7fa7c8b84700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa7a806ace0 0x7fa7a806d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8006fd0 tx=0x7fa7b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:59.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.370+0000 7fa7c17fa700 1 -- 172.21.15.22:0/4166226882 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa7b0057c50 con 0x7fa7c40ce860 2023-07-06T01:19:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.565+0000 7fa7cade8700 1 -- 172.21.15.22:0/4166226882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa7c406de90 con 0x7fa7c40ce860 2023-07-06T01:19:59.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.566+0000 7fa7c17fa700 1 -- 172.21.15.22:0/4166226882 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa7b00577e0 con 0x7fa7c40ce860 2023-07-06T01:19:59.569 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:19:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.568+0000 7fa7cade8700 1 -- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa7a806ace0 msgr2=0x7fa7a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.568+0000 7fa7cade8700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa7a806ace0 0x7fa7a806d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8006fd0 tx=0x7fa7b8008040 comp rx=0 tx=0).stop 2023-07-06T01:19:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.568+0000 7fa7cade8700 1 -- 172.21.15.22:0/4166226882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7c40ce860 msgr2=0x7fa7c40c47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.568+0000 7fa7cade8700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7c40ce860 0x7fa7c40c47c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000e810 tx=0x7fa7b000ebd0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.568+0000 7fa7cade8700 1 -- 172.21.15.22:0/4166226882 shutdown_connections 2023-07-06T01:19:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.568+0000 7fa7cade8700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7c415a710 0x7fa7c40c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.568+0000 7fa7cade8700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa7a806ace0 0x7fa7a806d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.568+0000 7fa7cade8700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7c40cbae0 0x7fa7c41ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.568+0000 7fa7cade8700 1 --2- 172.21.15.22:0/4166226882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7c40ce860 0x7fa7c40c47c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.568+0000 7fa7cade8700 1 -- 172.21.15.22:0/4166226882 >> 172.21.15.22:0/4166226882 conn(0x7fa7c40999f0 msgr2=0x7fa7c406e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.568+0000 7fa7cade8700 1 -- 172.21.15.22:0/4166226882 shutdown_connections 2023-07-06T01:19:59.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.568+0000 7fa7cade8700 1 -- 172.21.15.22:0/4166226882 wait complete. 2023-07-06T01:19:59.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:19:59.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.708+0000 7fc7aef97700 1 Processor -- start 2023-07-06T01:19:59.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.708+0000 7fc7aef97700 1 -- start start 2023-07-06T01:19:59.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.709+0000 7fc7aef97700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc7a8159360 0x7fc7a815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.709+0000 7fc7aef97700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7a80cc660 0x7fc7a80ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.709+0000 7fc7aef97700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a80cf230 con 0x7fc7a8070a20 2023-07-06T01:19:59.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.709+0000 7fc7aef97700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a815df20 con 0x7fc7a8159360 2023-07-06T01:19:59.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.709+0000 7fc7aef97700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80ccff0 con 0x7fc7a80cc660 2023-07-06T01:19:59.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.709+0000 7fc7acd33700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc7a8159360 0x7fc7a815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.709+0000 7fc7acd33700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc7a8159360 0x7fc7a815d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46504/0 (socket says 172.21.15.22:46504) 2023-07-06T01:19:59.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.709+0000 7fc7acd33700 1 -- 172.21.15.22:0/1318015811 learned_addr learned my addr 172.21.15.22:0/1318015811 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:59.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.710+0000 7fc7a7fff700 1 --2- 172.21.15.22:0/1318015811 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7a80cc660 0x7fc7a80ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.710+0000 7fc7acd33700 1 -- 172.21.15.22:0/1318015811 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7a80cc660 msgr2=0x7fc7a80ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:59.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.710+0000 7fc7acd33700 1 --2- 172.21.15.22:0/1318015811 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7a80cc660 0x7fc7a80ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.710+0000 7fc7acd33700 1 -- 172.21.15.22:0/1318015811 >> v1:172.21.15.103:6789/0 conn(0x7fc7a8070a20 legacy=0x7fc7a8070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:19:59.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.710+0000 7fc7acd33700 1 -- 172.21.15.22:0/1318015811 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a80ce620 con 0x7fc7a8159360 2023-07-06T01:19:59.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.710+0000 7fc7acd33700 1 --2- 172.21.15.22:0/1318015811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7a8159360 0x7fc7a815d9e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000d150 tx=0x7fc7a000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9bf70860811bae server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:59.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.710+0000 7fc7a77fe700 1 -- 172.21.15.22:0/1318015811 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0010070 con 0x7fc7a8159360 2023-07-06T01:19:59.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.710+0000 7fc7a77fe700 1 -- 172.21.15.22:0/1318015811 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a000dd80 con 0x7fc7a8159360 2023-07-06T01:19:59.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.710+0000 7fc7a77fe700 1 -- 172.21.15.22:0/1318015811 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a000b310 con 0x7fc7a8159360 2023-07-06T01:19:59.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.711+0000 7fc7aef97700 1 -- 172.21.15.22:0/1318015811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7a8159360 msgr2=0x7fc7a815d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.711+0000 7fc7aef97700 1 --2- 172.21.15.22:0/1318015811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7a8159360 0x7fc7a815d9e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000d150 tx=0x7fc7a000d460 comp rx=0 tx=0).stop 2023-07-06T01:19:59.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.711+0000 7fc7aef97700 1 -- 172.21.15.22:0/1318015811 shutdown_connections 2023-07-06T01:19:59.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.711+0000 7fc7aef97700 1 --2- 172.21.15.22:0/1318015811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7a8159360 0x7fc7a815d9e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.711+0000 7fc7aef97700 1 --2- 172.21.15.22:0/1318015811 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7a80cc660 0x7fc7a80ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.711+0000 7fc7aef97700 1 -- 172.21.15.22:0/1318015811 >> 172.21.15.22:0/1318015811 conn(0x7fc7a80b1890 msgr2=0x7fc7a80b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:59.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.711+0000 7fc7aef97700 1 -- 172.21.15.22:0/1318015811 shutdown_connections 2023-07-06T01:19:59.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.711+0000 7fc7aef97700 1 -- 172.21.15.22:0/1318015811 wait complete. 2023-07-06T01:19:59.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.712+0000 7fc7aef97700 1 Processor -- start 2023-07-06T01:19:59.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.712+0000 7fc7aef97700 1 -- start start 2023-07-06T01:19:59.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.713+0000 7fc7aef97700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7a8070a20 0x7fc7a815bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.713+0000 7fc7aef97700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7a80cc660 0x7fc7a815a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.713+0000 7fc7aef97700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7a8159360 0x7fc7a80c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.713+0000 7fc7aef97700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80b5250 con 0x7fc7a8070a20 2023-07-06T01:19:59.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.713+0000 7fc7aef97700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80b3cd0 con 0x7fc7a8159360 2023-07-06T01:19:59.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.713+0000 7fc7aef97700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80b2c30 con 0x7fc7a80cc660 2023-07-06T01:19:59.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.713+0000 7fc7acd33700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7a8070a20 0x7fc7a815bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7acd33700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7a8070a20 0x7fc7a815bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46510/0 (socket says 172.21.15.22:46510) 2023-07-06T01:19:59.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7acd33700 1 -- 172.21.15.22:0/2661947210 learned_addr learned my addr 172.21.15.22:0/2661947210 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:19:59.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7ad534700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7a8159360 0x7fc7a80c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7a7fff700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7a80cc660 0x7fc7a815a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7acd33700 1 -- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7a80cc660 msgr2=0x7fc7a815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7acd33700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7a80cc660 0x7fc7a815a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7acd33700 1 -- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7a8159360 msgr2=0x7fc7a80c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7acd33700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7a8159360 0x7fc7a80c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7acd33700 1 -- 172.21.15.22:0/2661947210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a000bd40 con 0x7fc7a8070a20 2023-07-06T01:19:59.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7a7fff700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7a80cc660 0x7fc7a815a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:59.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7ad534700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7a8159360 0x7fc7a80c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:19:59.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7acd33700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7a8070a20 0x7fc7a815bf40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0017040 tx=0x7fc7a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:59.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7a57fa700 1 -- 172.21.15.22:0/2661947210 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a000aba0 con 0x7fc7a8070a20 2023-07-06T01:19:59.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7a57fa700 1 -- 172.21.15.22:0/2661947210 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a000ad40 con 0x7fc7a8070a20 2023-07-06T01:19:59.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7aef97700 1 -- 172.21.15.22:0/2661947210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a80b2f10 con 0x7fc7a8070a20 2023-07-06T01:19:59.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7a57fa700 1 -- 172.21.15.22:0/2661947210 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0004270 con 0x7fc7a8070a20 2023-07-06T01:19:59.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.714+0000 7fc7aef97700 1 -- 172.21.15.22:0/2661947210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a80b34d0 con 0x7fc7a8070a20 2023-07-06T01:19:59.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.715+0000 7fc7aef97700 1 -- 172.21.15.22:0/2661947210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a80744b0 con 0x7fc7a8070a20 2023-07-06T01:19:59.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.716+0000 7fc7a57fa700 1 -- 172.21.15.22:0/2661947210 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc7a00043d0 con 0x7fc7a8070a20 2023-07-06T01:19:59.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.716+0000 7fc7a57fa700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc78806ace0 0x7fc78806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:19:59.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.716+0000 7fc7a57fa700 1 -- 172.21.15.22:0/2661947210 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc7a008ccf0 con 0x7fc7a8070a20 2023-07-06T01:19:59.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.716+0000 7fc7a7fff700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc78806ace0 0x7fc78806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:19:59.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.717+0000 7fc7a7fff700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc78806ace0 0x7fc78806d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc790006fd0 tx=0x7fc790008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:19:59.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.720+0000 7fc7a57fa700 1 -- 172.21.15.22:0/2661947210 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc7a0091070 con 0x7fc7a8070a20 2023-07-06T01:19:59.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.916+0000 7fc7aef97700 1 -- 172.21.15.22:0/2661947210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7a80cddc0 con 0x7fc7a8070a20 2023-07-06T01:19:59.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.916+0000 7fc7a57fa700 1 -- 172.21.15.22:0/2661947210 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7a0057e30 con 0x7fc7a8070a20 2023-07-06T01:19:59.919 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:19:59.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.918+0000 7fc7aef97700 1 -- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc78806ace0 msgr2=0x7fc78806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.918+0000 7fc7aef97700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc78806ace0 0x7fc78806d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc790006fd0 tx=0x7fc790008040 comp rx=0 tx=0).stop 2023-07-06T01:19:59.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.918+0000 7fc7aef97700 1 -- 172.21.15.22:0/2661947210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7a8070a20 msgr2=0x7fc7a815bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:19:59.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.918+0000 7fc7aef97700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7a8070a20 0x7fc7a815bf40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0017040 tx=0x7fc7a0004ad0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.919+0000 7fc7aef97700 1 -- 172.21.15.22:0/2661947210 shutdown_connections 2023-07-06T01:19:59.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.919+0000 7fc7aef97700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc78806ace0 0x7fc78806d190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.919+0000 7fc7aef97700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7a8159360 0x7fc7a80c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.919+0000 7fc7aef97700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc7a8070a20 0x7fc7a815bf40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.919+0000 7fc7aef97700 1 --2- 172.21.15.22:0/2661947210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7a80cc660 0x7fc7a815a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:19:59.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.919+0000 7fc7aef97700 1 -- 172.21.15.22:0/2661947210 >> 172.21.15.22:0/2661947210 conn(0x7fc7a80b1890 msgr2=0x7fc7a80b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:19:59.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.919+0000 7fc7aef97700 1 -- 172.21.15.22:0/2661947210 shutdown_connections 2023-07-06T01:19:59.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:19:59.919+0000 7fc7aef97700 1 -- 172.21.15.22:0/2661947210 wait complete. 2023-07-06T01:19:59.933 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T01:20:00.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.059+0000 7fe407de2700 1 Processor -- start 2023-07-06T01:20:00.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.059+0000 7fe407de2700 1 -- start start 2023-07-06T01:20:00.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe407de2700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe400159360 0x7fe40015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe407de2700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe4000cc660 0x7fe4000ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe407de2700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4000cf230 con 0x7fe400070a20 2023-07-06T01:20:00.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe407de2700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40015df20 con 0x7fe4000cc660 2023-07-06T01:20:00.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe407de2700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4000cd020 con 0x7fe400159360 2023-07-06T01:20:00.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe40537d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe4000cc660 0x7fe4000ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe40537d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe4000cc660 0x7fe4000ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46518/0 (socket says 172.21.15.22:46518) 2023-07-06T01:20:00.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe40537d700 1 -- 172.21.15.22:0/4283403848 learned_addr learned my addr 172.21.15.22:0/4283403848 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:00.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe405b7e700 1 --2- 172.21.15.22:0/4283403848 >> v2:172.21.15.103:3300/0 conn(0x7fe400159360 0x7fe40015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe40537d700 1 -- 172.21.15.22:0/4283403848 >> v2:172.21.15.103:3300/0 conn(0x7fe400159360 msgr2=0x7fe40015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:00.064 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe40537d700 1 --2- 172.21.15.22:0/4283403848 >> v2:172.21.15.103:3300/0 conn(0x7fe400159360 0x7fe40015d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe40537d700 1 -- 172.21.15.22:0/4283403848 >> v1:172.21.15.103:6789/0 conn(0x7fe400070a20 legacy=0x7fe400070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe40537d700 1 -- 172.21.15.22:0/4283403848 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4000ce620 con 0x7fe4000cc660 2023-07-06T01:20:00.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe40537d700 1 --2- 172.21.15.22:0/4283403848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4000cc660 0x7fe4000ccae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe400071aa0 tx=0x7fe3f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09275c53c9d4060 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:00.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe404b7c700 1 -- 172.21.15.22:0/4283403848 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f000b870 con 0x7fe4000cc660 2023-07-06T01:20:00.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe404b7c700 1 -- 172.21.15.22:0/4283403848 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f000b9d0 con 0x7fe4000cc660 2023-07-06T01:20:00.065 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.060+0000 7fe404b7c700 1 -- 172.21.15.22:0/4283403848 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f000bd60 con 0x7fe4000cc660 2023-07-06T01:20:00.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.061+0000 7fe407de2700 1 -- 172.21.15.22:0/4283403848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4000cc660 msgr2=0x7fe4000ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.061+0000 7fe407de2700 1 --2- 172.21.15.22:0/4283403848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4000cc660 0x7fe4000ccae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe400071aa0 tx=0x7fe3f0009bd0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.061+0000 7fe407de2700 1 -- 172.21.15.22:0/4283403848 shutdown_connections 2023-07-06T01:20:00.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.061+0000 7fe407de2700 1 --2- 172.21.15.22:0/4283403848 >> v2:172.21.15.103:3300/0 conn(0x7fe400159360 0x7fe40015d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.068 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.061+0000 7fe407de2700 1 --2- 172.21.15.22:0/4283403848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4000cc660 0x7fe4000ccae0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.061+0000 7fe407de2700 1 -- 172.21.15.22:0/4283403848 >> 172.21.15.22:0/4283403848 conn(0x7fe4000b1890 msgr2=0x7fe4000b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:00.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.061+0000 7fe407de2700 1 -- 172.21.15.22:0/4283403848 shutdown_connections 2023-07-06T01:20:00.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.061+0000 7fe407de2700 1 -- 172.21.15.22:0/4283403848 wait complete. 2023-07-06T01:20:00.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.062+0000 7fe407de2700 1 Processor -- start 2023-07-06T01:20:00.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.062+0000 7fe407de2700 1 -- start start 2023-07-06T01:20:00.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe407de2700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe400070a20 0x7fe40014ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe407de2700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4000cc660 0x7fe400159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.069 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe407de2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe400159360 0x7fe4001ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe407de2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40015a140 con 0x7fe400159360 2023-07-06T01:20:00.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe407de2700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000b3820 con 0x7fe400070a20 2023-07-06T01:20:00.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe407de2700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40014e330 con 0x7fe4000cc660 2023-07-06T01:20:00.070 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe40637f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe400159360 0x7fe4001ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe40637f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe400159360 0x7fe4001ef2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46528/0 (socket says 172.21.15.22:46528) 2023-07-06T01:20:00.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe40637f700 1 -- 172.21.15.22:0/4150301211 learned_addr learned my addr 172.21.15.22:0/4150301211 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:00.071 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe405b7e700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe400070a20 0x7fe40014ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe40537d700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4000cc660 0x7fe400159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe40637f700 1 -- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4000cc660 msgr2=0x7fe400159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe40637f700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4000cc660 0x7fe400159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe40637f700 1 -- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe400070a20 msgr2=0x7fe40014ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.072 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe40637f700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe400070a20 0x7fe40014ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.063+0000 7fe40637f700 1 -- 172.21.15.22:0/4150301211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f0009550 con 0x7fe400159360 2023-07-06T01:20:00.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.064+0000 7fe40637f700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe400159360 0x7fe4001ef2e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800cc50 tx=0x7fe3f800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:00.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.064+0000 7fe3f67fc700 1 -- 172.21.15.22:0/4150301211 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f800ad80 con 0x7fe400159360 2023-07-06T01:20:00.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.064+0000 7fe3f67fc700 1 -- 172.21.15.22:0/4150301211 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f800ce50 con 0x7fe400159360 2023-07-06T01:20:00.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.064+0000 7fe407de2700 1 -- 172.21.15.22:0/4150301211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40015a390 con 0x7fe400159360 2023-07-06T01:20:00.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.064+0000 7fe3f67fc700 1 -- 172.21.15.22:0/4150301211 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8003c40 con 0x7fe400159360 2023-07-06T01:20:00.073 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.064+0000 7fe407de2700 1 -- 172.21.15.22:0/4150301211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4000b3a70 con 0x7fe400159360 2023-07-06T01:20:00.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.065+0000 7fe407de2700 1 -- 172.21.15.22:0/4150301211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40015a8b0 con 0x7fe400159360 2023-07-06T01:20:00.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.065+0000 7fe3f67fc700 1 -- 172.21.15.22:0/4150301211 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe3f80050d0 con 0x7fe400159360 2023-07-06T01:20:00.074 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.066+0000 7fe3f67fc700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe3e406ace0 0x7fe3e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.066+0000 7fe3f67fc700 1 -- 172.21.15.22:0/4150301211 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3f8089540 con 0x7fe400159360 2023-07-06T01:20:00.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.066+0000 7fe405b7e700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe3e406ace0 0x7fe3e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.066+0000 7fe405b7e700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe3e406ace0 0x7fe3e406d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00b9b0 tx=0x7fe3fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:00.075 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.070+0000 7fe3f67fc700 1 -- 172.21.15.22:0/4150301211 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe3f8054680 con 0x7fe400159360 2023-07-06T01:20:00.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.270+0000 7fe407de2700 1 -- 172.21.15.22:0/4150301211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe4000744b0 con 0x7fe400159360 2023-07-06T01:20:00.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.270+0000 7fe3f67fc700 1 -- 172.21.15.22:0/4150301211 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe3f8057ca0 con 0x7fe400159360 2023-07-06T01:20:00.273 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-07-06T01:20:00.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.272+0000 7fe407de2700 1 -- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe3e406ace0 msgr2=0x7fe3e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.272+0000 7fe407de2700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe3e406ace0 0x7fe3e406d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00b9b0 tx=0x7fe3fc008040 comp rx=0 tx=0).stop 2023-07-06T01:20:00.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.272+0000 7fe407de2700 1 -- 172.21.15.22:0/4150301211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe400159360 msgr2=0x7fe4001ef2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.272+0000 7fe407de2700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe400159360 0x7fe4001ef2e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800cc50 tx=0x7fe3f800a340 comp rx=0 tx=0).stop 2023-07-06T01:20:00.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.272+0000 7fe407de2700 1 -- 172.21.15.22:0/4150301211 shutdown_connections 2023-07-06T01:20:00.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.272+0000 7fe407de2700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe400159360 0x7fe4001ef2e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.272+0000 7fe407de2700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe3e406ace0 0x7fe3e406d190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.272+0000 7fe407de2700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe400070a20 0x7fe40014ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.272+0000 7fe407de2700 1 --2- 172.21.15.22:0/4150301211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4000cc660 0x7fe400159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.272+0000 7fe407de2700 1 -- 172.21.15.22:0/4150301211 >> 172.21.15.22:0/4150301211 conn(0x7fe4000b1890 msgr2=0x7fe400158490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:00.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.272+0000 7fe407de2700 1 -- 172.21.15.22:0/4150301211 shutdown_connections 2023-07-06T01:20:00.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.272+0000 7fe407de2700 1 -- 172.21.15.22:0/4150301211 wait complete. 2023-07-06T01:20:00.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T01:20:00.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.414+0000 7f14b0a59700 1 Processor -- start 2023-07-06T01:20:00.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.414+0000 7f14b0a59700 1 -- start start 2023-07-06T01:20:00.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14b0a59700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f14ac1597a0 0x7f14ac06cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14b0a59700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f14ac0cc750 0x7f14ac0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14b0a59700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14ac0c9a50 0x7f14ac0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14b0a59700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac0ca410 con 0x7f14ac1597a0 2023-07-06T01:20:00.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14b0a59700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac0ccf30 con 0x7f14ac0cc750 2023-07-06T01:20:00.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14b0a59700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac0cd210 con 0x7f14ac0c9a50 2023-07-06T01:20:00.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14aa59c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f14ac1597a0 0x7f14ac06cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14aa59c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f14ac1597a0 0x7f14ac06cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46532/0 (socket says 172.21.15.22:46532) 2023-07-06T01:20:00.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14aa59c700 1 -- 172.21.15.22:0/2542889346 learned_addr learned my addr 172.21.15.22:0/2542889346 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:00.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14a9d9b700 1 --2- 172.21.15.22:0/2542889346 >> v2:172.21.15.103:3300/0 conn(0x7f14ac0cc750 0x7f14ac0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14aad9d700 1 --2- 172.21.15.22:0/2542889346 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14ac0c9a50 0x7f14ac0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14aa59c700 1 -- 172.21.15.22:0/2542889346 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14ac0c9a50 msgr2=0x7f14ac0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:00.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14aa59c700 1 --2- 172.21.15.22:0/2542889346 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14ac0c9a50 0x7f14ac0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14aa59c700 1 -- 172.21.15.22:0/2542889346 >> v2:172.21.15.103:3300/0 conn(0x7f14ac0cc750 msgr2=0x7f14ac0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:00.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14aa59c700 1 --2- 172.21.15.22:0/2542889346 >> v2:172.21.15.103:3300/0 conn(0x7f14ac0cc750 0x7f14ac0cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14aa59c700 1 -- 172.21.15.22:0/2542889346 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac0cd4f0 con 0x7f14ac1597a0 2023-07-06T01:20:00.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14aa59c700 1 --2- 172.21.15.22:0/2542889346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ac1597a0 0x7f14ac06cf20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f14ac068270 tx=0x7f149c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db7b51d98cb21ff server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:00.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.415+0000 7f14a959a700 1 -- 172.21.15.22:0/2542889346 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00b870 con 0x7f14ac1597a0 2023-07-06T01:20:00.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.416+0000 7f14a959a700 1 -- 172.21.15.22:0/2542889346 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c00b9d0 con 0x7f14ac1597a0 2023-07-06T01:20:00.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.416+0000 7f14a959a700 1 -- 172.21.15.22:0/2542889346 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00bd60 con 0x7f14ac1597a0 2023-07-06T01:20:00.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.416+0000 7f14b0a59700 1 -- 172.21.15.22:0/2542889346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ac1597a0 msgr2=0x7f14ac06cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.416+0000 7f14b0a59700 1 --2- 172.21.15.22:0/2542889346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ac1597a0 0x7f14ac06cf20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f14ac068270 tx=0x7f149c009bd0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.416+0000 7f14b0a59700 1 -- 172.21.15.22:0/2542889346 shutdown_connections 2023-07-06T01:20:00.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.416+0000 7f14b0a59700 1 --2- 172.21.15.22:0/2542889346 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14ac0c9a50 0x7f14ac0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.416+0000 7f14b0a59700 1 --2- 172.21.15.22:0/2542889346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ac1597a0 0x7f14ac06cf20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.416+0000 7f14b0a59700 1 --2- 172.21.15.22:0/2542889346 >> v2:172.21.15.103:3300/0 conn(0x7f14ac0cc750 0x7f14ac0cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.416+0000 7f14b0a59700 1 -- 172.21.15.22:0/2542889346 >> 172.21.15.22:0/2542889346 conn(0x7f14ac0b0710 msgr2=0x7f14ac0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:00.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.416+0000 7f14b0a59700 1 -- 172.21.15.22:0/2542889346 shutdown_connections 2023-07-06T01:20:00.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.416+0000 7f14b0a59700 1 -- 172.21.15.22:0/2542889346 wait complete. 2023-07-06T01:20:00.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.418+0000 7f14b0a59700 1 Processor -- start 2023-07-06T01:20:00.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.418+0000 7f14b0a59700 1 -- start start 2023-07-06T01:20:00.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.418+0000 7f14b0a59700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14ac0c9a50 0x7f14ac155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.418+0000 7f14b0a59700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ac0cc750 0x7f14ac155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.418+0000 7f14b0a59700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14ac1597a0 0x7f14ac0ad430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.418+0000 7f14b0a59700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac0b1ab0 con 0x7f14ac0cc750 2023-07-06T01:20:00.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.418+0000 7f14b0a59700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac155550 con 0x7f14ac0c9a50 2023-07-06T01:20:00.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.418+0000 7f14b0a59700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac070970 con 0x7f14ac1597a0 2023-07-06T01:20:00.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.418+0000 7f14a9d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ac0cc750 0x7f14ac155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.418+0000 7f14a9d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ac0cc750 0x7f14ac155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46540/0 (socket says 172.21.15.22:46540) 2023-07-06T01:20:00.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.418+0000 7f14a9d9b700 1 -- 172.21.15.22:0/599123860 learned_addr learned my addr 172.21.15.22:0/599123860 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:00.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.419+0000 7f14aa59c700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14ac0c9a50 0x7f14ac155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.419+0000 7f14aad9d700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14ac1597a0 0x7f14ac0ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.419+0000 7f14a9d9b700 1 -- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14ac1597a0 msgr2=0x7f14ac0ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.419+0000 7f14a9d9b700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14ac1597a0 0x7f14ac0ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.419+0000 7f14a9d9b700 1 -- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14ac0c9a50 msgr2=0x7f14ac155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.419+0000 7f14a9d9b700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14ac0c9a50 0x7f14ac155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.419+0000 7f14a9d9b700 1 -- 172.21.15.22:0/599123860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c009550 con 0x7f14ac0cc750 2023-07-06T01:20:00.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.419+0000 7f14a9d9b700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ac0cc750 0x7f14ac155010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e810 tx=0x7f14a000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:00.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.419+0000 7f149affd700 1 -- 172.21.15.22:0/599123860 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0013610 con 0x7f14ac0cc750 2023-07-06T01:20:00.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.419+0000 7f149affd700 1 -- 172.21.15.22:0/599123860 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a0013770 con 0x7f14ac0cc750 2023-07-06T01:20:00.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.419+0000 7f14b0a59700 1 -- 172.21.15.22:0/599123860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ac0b1d50 con 0x7f14ac0cc750 2023-07-06T01:20:00.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.419+0000 7f149affd700 1 -- 172.21.15.22:0/599123860 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0013b20 con 0x7f14ac0cc750 2023-07-06T01:20:00.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.419+0000 7f14b0a59700 1 -- 172.21.15.22:0/599123860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac070c50 con 0x7f14ac0cc750 2023-07-06T01:20:00.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.420+0000 7f14b0a59700 1 -- 172.21.15.22:0/599123860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ac0744b0 con 0x7f14ac0cc750 2023-07-06T01:20:00.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.421+0000 7f149affd700 1 -- 172.21.15.22:0/599123860 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f14a0013c80 con 0x7f14ac0cc750 2023-07-06T01:20:00.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.421+0000 7f149affd700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f148c06ace0 0x7f148c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.421+0000 7f149affd700 1 -- 172.21.15.22:0/599123860 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f14a00890c0 con 0x7f14ac0cc750 2023-07-06T01:20:00.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.421+0000 7f14aa59c700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f148c06ace0 0x7f148c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.422+0000 7f14aa59c700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f148c06ace0 0x7f148c06d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f149c000c00 tx=0x7f149c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:00.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.425+0000 7f149affd700 1 -- 172.21.15.22:0/599123860 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f14a0054180 con 0x7f14ac0cc750 2023-07-06T01:20:00.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.621+0000 7f14b0a59700 1 -- 172.21.15.22:0/599123860 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f14ac0b1500 con 0x7f14ac0cc750 2023-07-06T01:20:00.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.621+0000 7f149affd700 1 -- 172.21.15.22:0/599123860 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f14a00577a0 con 0x7f14ac0cc750 2023-07-06T01:20:00.624 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-07-06T01:20:00.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.624+0000 7f14b0a59700 1 -- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f148c06ace0 msgr2=0x7f148c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.624+0000 7f14b0a59700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f148c06ace0 0x7f148c06d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f149c000c00 tx=0x7f149c008030 comp rx=0 tx=0).stop 2023-07-06T01:20:00.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.624+0000 7f14b0a59700 1 -- 172.21.15.22:0/599123860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ac0cc750 msgr2=0x7f14ac155010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.624+0000 7f14b0a59700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ac0cc750 0x7f14ac155010 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e810 tx=0x7f14a000ebd0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.624+0000 7f14b0a59700 1 -- 172.21.15.22:0/599123860 shutdown_connections 2023-07-06T01:20:00.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.624+0000 7f14b0a59700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14ac1597a0 0x7f14ac0ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.624+0000 7f14b0a59700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f148c06ace0 0x7f148c06d190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.624+0000 7f14b0a59700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14ac0c9a50 0x7f14ac155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.624+0000 7f14b0a59700 1 --2- 172.21.15.22:0/599123860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14ac0cc750 0x7f14ac155010 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.624+0000 7f14b0a59700 1 -- 172.21.15.22:0/599123860 >> 172.21.15.22:0/599123860 conn(0x7f14ac0b0710 msgr2=0x7f14ac06c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.624+0000 7f14b0a59700 1 -- 172.21.15.22:0/599123860 shutdown_connections 2023-07-06T01:20:00.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.624+0000 7f14b0a59700 1 -- 172.21.15.22:0/599123860 wait complete. 2023-07-06T01:20:00.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T01:20:00.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.763+0000 7f054c8ce700 1 Processor -- start 2023-07-06T01:20:00.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.763+0000 7f054c8ce700 1 -- start start 2023-07-06T01:20:00.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f054c8ce700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0548159870 0x7f054815df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f054c8ce700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f05480715f0 0x7f05480cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f054c8ce700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05480ce550 con 0x7f05480d0130 2023-07-06T01:20:00.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f054c8ce700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05480d05b0 con 0x7f0548159870 2023-07-06T01:20:00.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f054c8ce700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05480cdb60 con 0x7f05480715f0 2023-07-06T01:20:00.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f054659c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0548159870 0x7f054815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f0546d9d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f05480d0130 0x7f05480cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37292/0 (socket says 172.21.15.22:37292) 2023-07-06T01:20:00.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f0546d9d700 1 -- 172.21.15.22:0/2024840083 learned_addr learned my addr 172.21.15.22:0/2024840083 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:00.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f0545d9b700 1 --2- 172.21.15.22:0/2024840083 >> v2:172.21.15.103:3300/0 conn(0x7f05480715f0 0x7f05480cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f054659c700 1 -- 172.21.15.22:0/2024840083 >> v2:172.21.15.103:3300/0 conn(0x7f05480715f0 msgr2=0x7f05480cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:00.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f054659c700 1 --2- 172.21.15.22:0/2024840083 >> v2:172.21.15.103:3300/0 conn(0x7f05480715f0 0x7f05480cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f054659c700 1 -- 172.21.15.22:0/2024840083 >> v1:172.21.15.22:6789/0 conn(0x7f05480d0130 legacy=0x7f05480cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f054659c700 1 -- 172.21.15.22:0/2024840083 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054815d490 con 0x7f0548159870 2023-07-06T01:20:00.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.764+0000 7f054559a700 1 -- 172.21.15.22:0/2024840083 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004409251 0 0) 0x7f05480ce550 con 0x7f05480d0130 2023-07-06T01:20:00.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.765+0000 7f054659c700 1 --2- 172.21.15.22:0/2024840083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0548159870 0x7f054815df70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f05480cd780 tx=0x7f053c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b67cdb6ea487670 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:00.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.765+0000 7f054559a700 1 -- 172.21.15.22:0/2024840083 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c011830 con 0x7f0548159870 2023-07-06T01:20:00.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.765+0000 7f054559a700 1 -- 172.21.15.22:0/2024840083 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c011990 con 0x7f0548159870 2023-07-06T01:20:00.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.765+0000 7f054559a700 1 -- 172.21.15.22:0/2024840083 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c011d20 con 0x7f0548159870 2023-07-06T01:20:00.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.765+0000 7f054c8ce700 1 -- 172.21.15.22:0/2024840083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0548159870 msgr2=0x7f054815df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.765+0000 7f054c8ce700 1 --2- 172.21.15.22:0/2024840083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0548159870 0x7f054815df70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f05480cd780 tx=0x7f053c009b20 comp rx=0 tx=0).stop 2023-07-06T01:20:00.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.765+0000 7f054c8ce700 1 -- 172.21.15.22:0/2024840083 shutdown_connections 2023-07-06T01:20:00.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.765+0000 7f054c8ce700 1 --2- 172.21.15.22:0/2024840083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0548159870 0x7f054815df70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.765+0000 7f054c8ce700 1 --2- 172.21.15.22:0/2024840083 >> v2:172.21.15.103:3300/0 conn(0x7f05480715f0 0x7f05480cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.765+0000 7f054c8ce700 1 -- 172.21.15.22:0/2024840083 >> 172.21.15.22:0/2024840083 conn(0x7f054806b110 msgr2=0x7f054806b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:00.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.765+0000 7f054c8ce700 1 -- 172.21.15.22:0/2024840083 shutdown_connections 2023-07-06T01:20:00.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.765+0000 7f054c8ce700 1 -- 172.21.15.22:0/2024840083 wait complete. 2023-07-06T01:20:00.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.767+0000 7f054c8ce700 1 Processor -- start 2023-07-06T01:20:00.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.767+0000 7f054c8ce700 1 -- start start 2023-07-06T01:20:00.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f054c8ce700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05480715f0 0x7f0548154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f054c8ce700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05480d0130 0x7f0548154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f054c8ce700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0548159870 0x7f05480c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f054c8ce700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05480c6e60 con 0x7f0548159870 2023-07-06T01:20:00.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f054c8ce700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05480c7140 con 0x7f05480d0130 2023-07-06T01:20:00.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f054c8ce700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05480c7420 con 0x7f05480715f0 2023-07-06T01:20:00.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f0546d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0548159870 0x7f05480c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f0546d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0548159870 0x7f05480c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46554/0 (socket says 172.21.15.22:46554) 2023-07-06T01:20:00.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f0546d9d700 1 -- 172.21.15.22:0/2922505825 learned_addr learned my addr 172.21.15.22:0/2922505825 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:00.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f0545d9b700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05480d0130 0x7f0548154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f054659c700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05480715f0 0x7f0548154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f0546d9d700 1 -- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05480715f0 msgr2=0x7f0548154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f0546d9d700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05480715f0 0x7f0548154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f0546d9d700 1 -- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05480d0130 msgr2=0x7f0548154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f0546d9d700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05480d0130 0x7f0548154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f0546d9d700 1 -- 172.21.15.22:0/2922505825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c0094a0 con 0x7f0548159870 2023-07-06T01:20:00.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f0546d9d700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0548159870 0x7f05480c7f00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0540002230 tx=0x7f054000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:00.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f0536ffd700 1 -- 172.21.15.22:0/2922505825 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054000aac0 con 0x7f0548159870 2023-07-06T01:20:00.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f0536ffd700 1 -- 172.21.15.22:0/2922505825 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054000ac40 con 0x7f0548159870 2023-07-06T01:20:00.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f054c8ce700 1 -- 172.21.15.22:0/2922505825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05480c7700 con 0x7f0548159870 2023-07-06T01:20:00.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.768+0000 7f0536ffd700 1 -- 172.21.15.22:0/2922505825 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400070d0 con 0x7f0548159870 2023-07-06T01:20:00.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.769+0000 7f054c8ce700 1 -- 172.21.15.22:0/2922505825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05480c9940 con 0x7f0548159870 2023-07-06T01:20:00.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.769+0000 7f054c8ce700 1 -- 172.21.15.22:0/2922505825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05480744b0 con 0x7f0548159870 2023-07-06T01:20:00.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.770+0000 7f0536ffd700 1 -- 172.21.15.22:0/2922505825 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0540007250 con 0x7f0548159870 2023-07-06T01:20:00.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.770+0000 7f0536ffd700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f052806ace0 0x7f052806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:00.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.770+0000 7f0536ffd700 1 -- 172.21.15.22:0/2922505825 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f05400891d0 con 0x7f0548159870 2023-07-06T01:20:00.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.771+0000 7f054659c700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f052806ace0 0x7f052806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:00.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.771+0000 7f054659c700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f052806ace0 0x7f052806d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f053c0090b0 tx=0x7f053c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:00.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.775+0000 7f0536ffd700 1 -- 172.21.15.22:0/2922505825 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0540054310 con 0x7f0548159870 2023-07-06T01:20:00.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.988+0000 7f054c8ce700 1 -- 172.21.15.22:0/2922505825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0548159090 con 0x7f0548159870 2023-07-06T01:20:00.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.988+0000 7f0536ffd700 1 -- 172.21.15.22:0/2922505825 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0540057930 con 0x7f0548159870 2023-07-06T01:20:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.990+0000 7f054c8ce700 1 -- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f052806ace0 msgr2=0x7f052806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.990+0000 7f054c8ce700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f052806ace0 0x7f052806d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f053c0090b0 tx=0x7f053c01c040 comp rx=0 tx=0).stop 2023-07-06T01:20:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.990+0000 7f054c8ce700 1 -- 172.21.15.22:0/2922505825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0548159870 msgr2=0x7f05480c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.990+0000 7f054c8ce700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0548159870 0x7f05480c7f00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0540002230 tx=0x7f054000cf50 comp rx=0 tx=0).stop 2023-07-06T01:20:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.990+0000 7f054c8ce700 1 -- 172.21.15.22:0/2922505825 shutdown_connections 2023-07-06T01:20:00.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.990+0000 7f054c8ce700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0548159870 0x7f05480c7f00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.990+0000 7f054c8ce700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05480715f0 0x7f0548154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.990+0000 7f054c8ce700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f052806ace0 0x7f052806d190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.990+0000 7f054c8ce700 1 --2- 172.21.15.22:0/2922505825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05480d0130 0x7f0548154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:00.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.990+0000 7f054c8ce700 1 -- 172.21.15.22:0/2922505825 >> 172.21.15.22:0/2922505825 conn(0x7f054806b110 msgr2=0x7f05480cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:00.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.990+0000 7f054c8ce700 1 -- 172.21.15.22:0/2922505825 shutdown_connections 2023-07-06T01:20:00.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:00.990+0000 7f054c8ce700 1 -- 172.21.15.22:0/2922505825 wait complete. 2023-07-06T01:20:00.997 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-07-06T01:20:01.005 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-06T01:20:01.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:01.005 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-07-06T01:20:01.042 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:20:01.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:01.043 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T01:20:01.108 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:20:01.108 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:20:01.130 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:20:01.130 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:20:01.190 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:20:01.191 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:20:01.251 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:20:01.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.374+0000 7f8f341ae700 1 Processor -- start 2023-07-06T01:20:01.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.375+0000 7f8f341ae700 1 -- start start 2023-07-06T01:20:01.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f341ae700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f2c15a610 0x7f8f2c06b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:01.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f341ae700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8f2c15eae0 0x7f8f2c06c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:01.380 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f341ae700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f2c0caa50 con 0x7f8f2c06c730 2023-07-06T01:20:01.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f341ae700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2c15a9e0 con 0x7f8f2c15eae0 2023-07-06T01:20:01.381 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f341ae700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c06cbb0 con 0x7f8f2c15a610 2023-07-06T01:20:01.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f31749700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8f2c15eae0 0x7f8f2c06c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:01.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f3274b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f8f2c06c730 0x7f8f2c06a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37304/0 (socket says 172.21.15.22:37304) 2023-07-06T01:20:01.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f3274b700 1 -- 172.21.15.22:0/2904272544 learned_addr learned my addr 172.21.15.22:0/2904272544 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:01.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f31f4a700 1 --2- 172.21.15.22:0/2904272544 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f2c15a610 0x7f8f2c06b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:01.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f31749700 1 -- 172.21.15.22:0/2904272544 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f2c15a610 msgr2=0x7f8f2c06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:01.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f31749700 1 --2- 172.21.15.22:0/2904272544 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f2c15a610 0x7f8f2c06b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f31749700 1 -- 172.21.15.22:0/2904272544 >> v1:172.21.15.22:6789/0 conn(0x7f8f2c06c730 legacy=0x7f8f2c06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:01.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f31749700 1 -- 172.21.15.22:0/2904272544 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c071f30 con 0x7f8f2c15eae0 2023-07-06T01:20:01.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f30f48700 1 -- 172.21.15.22:0/2904272544 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946138667 0 0) 0x7f8f2c0caa50 con 0x7f8f2c06c730 2023-07-06T01:20:01.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.376+0000 7f8f31749700 1 --2- 172.21.15.22:0/2904272544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c15eae0 0x7f8f2c06c1f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009840 tx=0x7f8f28009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffaa3ad8038d089 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:01.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.377+0000 7f8f30f48700 1 -- 172.21.15.22:0/2904272544 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f28011830 con 0x7f8f2c15eae0 2023-07-06T01:20:01.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.377+0000 7f8f30f48700 1 -- 172.21.15.22:0/2904272544 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f28011990 con 0x7f8f2c15eae0 2023-07-06T01:20:01.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.377+0000 7f8f30f48700 1 -- 172.21.15.22:0/2904272544 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f28011d20 con 0x7f8f2c15eae0 2023-07-06T01:20:01.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.377+0000 7f8f341ae700 1 -- 172.21.15.22:0/2904272544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c15eae0 msgr2=0x7f8f2c06c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:01.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.377+0000 7f8f341ae700 1 --2- 172.21.15.22:0/2904272544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c15eae0 0x7f8f2c06c1f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009840 tx=0x7f8f28009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:01.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.377+0000 7f8f341ae700 1 -- 172.21.15.22:0/2904272544 shutdown_connections 2023-07-06T01:20:01.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.377+0000 7f8f341ae700 1 --2- 172.21.15.22:0/2904272544 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f2c15a610 0x7f8f2c06b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.377+0000 7f8f341ae700 1 --2- 172.21.15.22:0/2904272544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c15eae0 0x7f8f2c06c1f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.377+0000 7f8f341ae700 1 -- 172.21.15.22:0/2904272544 >> 172.21.15.22:0/2904272544 conn(0x7f8f2c0ccb00 msgr2=0x7f8f2c0cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:01.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.377+0000 7f8f341ae700 1 -- 172.21.15.22:0/2904272544 shutdown_connections 2023-07-06T01:20:01.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.377+0000 7f8f341ae700 1 -- 172.21.15.22:0/2904272544 wait complete. 2023-07-06T01:20:01.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.378+0000 7f8f341ae700 1 Processor -- start 2023-07-06T01:20:01.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.379+0000 7f8f341ae700 1 -- start start 2023-07-06T01:20:01.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.379+0000 7f8f341ae700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f2c06c730 0x7f8f2c0c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:01.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.379+0000 7f8f341ae700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f2c15a610 0x7f8f2c0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:01.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.379+0000 7f8f341ae700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c15eae0 0x7f8f2c0afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:01.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.379+0000 7f8f341ae700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c155ff0 con 0x7f8f2c15eae0 2023-07-06T01:20:01.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.379+0000 7f8f341ae700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c1562d0 con 0x7f8f2c15a610 2023-07-06T01:20:01.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.379+0000 7f8f341ae700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c0b23c0 con 0x7f8f2c06c730 2023-07-06T01:20:01.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.379+0000 7f8f3274b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c15eae0 0x7f8f2c0afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:01.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.379+0000 7f8f3274b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c15eae0 0x7f8f2c0afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46566/0 (socket says 172.21.15.22:46566) 2023-07-06T01:20:01.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.379+0000 7f8f3274b700 1 -- 172.21.15.22:0/85148317 learned_addr learned my addr 172.21.15.22:0/85148317 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:01.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f31749700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f2c15a610 0x7f8f2c0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:01.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f31f4a700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f2c06c730 0x7f8f2c0c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:01.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f3274b700 1 -- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f2c06c730 msgr2=0x7f8f2c0c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:01.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f3274b700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f2c06c730 0x7f8f2c0c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f3274b700 1 -- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f2c15a610 msgr2=0x7f8f2c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:01.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f3274b700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f2c15a610 0x7f8f2c0af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f3274b700 1 -- 172.21.15.22:0/85148317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f280094a0 con 0x7f8f2c15eae0 2023-07-06T01:20:01.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f31f4a700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f2c06c730 0x7f8f2c0c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:01.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f3274b700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c15eae0 0x7f8f2c0afc20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f24002230 tx=0x7f8f2400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:01.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f227fc700 1 -- 172.21.15.22:0/85148317 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2400ac20 con 0x7f8f2c15eae0 2023-07-06T01:20:01.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f227fc700 1 -- 172.21.15.22:0/85148317 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2400ada0 con 0x7f8f2c15eae0 2023-07-06T01:20:01.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f341ae700 1 -- 172.21.15.22:0/85148317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2c0b26a0 con 0x7f8f2c15eae0 2023-07-06T01:20:01.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f227fc700 1 -- 172.21.15.22:0/85148317 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24007310 con 0x7f8f2c15eae0 2023-07-06T01:20:01.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.380+0000 7f8f341ae700 1 -- 172.21.15.22:0/85148317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2c155240 con 0x7f8f2c15eae0 2023-07-06T01:20:01.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.381+0000 7f8f341ae700 1 -- 172.21.15.22:0/85148317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f2c0b2220 con 0x7f8f2c15eae0 2023-07-06T01:20:01.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.382+0000 7f8f227fc700 1 -- 172.21.15.22:0/85148317 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8f24007470 con 0x7f8f2c15eae0 2023-07-06T01:20:01.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.382+0000 7f8f227fc700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8f1006ace0 0x7f8f1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:01.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.382+0000 7f8f227fc700 1 -- 172.21.15.22:0/85148317 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8f24089080 con 0x7f8f2c15eae0 2023-07-06T01:20:01.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.382+0000 7f8f31f4a700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8f1006ace0 0x7f8f1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:01.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.383+0000 7f8f31f4a700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8f1006ace0 0x7f8f1006d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c006fd0 tx=0x7f8f1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:01.391 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.387+0000 7f8f227fc700 1 -- 172.21.15.22:0/85148317 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8f2c0b2220 con 0x7f8f2c15eae0 2023-07-06T01:20:01.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.585+0000 7f8f341ae700 1 -- 172.21.15.22:0/85148317 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f2c0b2220 con 0x7f8f2c15eae0 2023-07-06T01:20:01.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.587+0000 7f8f227fc700 1 -- 172.21.15.22:0/85148317 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7f8f2c0b2220 con 0x7f8f2c15eae0 2023-07-06T01:20:01.591 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:01.592 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:20:01.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.589+0000 7f8f341ae700 1 -- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8f1006ace0 msgr2=0x7f8f1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:01.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.589+0000 7f8f341ae700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8f1006ace0 0x7f8f1006d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c006fd0 tx=0x7f8f1c008040 comp rx=0 tx=0).stop 2023-07-06T01:20:01.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.589+0000 7f8f341ae700 1 -- 172.21.15.22:0/85148317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c15eae0 msgr2=0x7f8f2c0afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:01.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.589+0000 7f8f341ae700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c15eae0 0x7f8f2c0afc20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f24002230 tx=0x7f8f2400a060 comp rx=0 tx=0).stop 2023-07-06T01:20:01.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.589+0000 7f8f341ae700 1 -- 172.21.15.22:0/85148317 shutdown_connections 2023-07-06T01:20:01.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.589+0000 7f8f341ae700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f2c15eae0 0x7f8f2c0afc20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.589+0000 7f8f341ae700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f2c06c730 0x7f8f2c0c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.589+0000 7f8f341ae700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8f1006ace0 0x7f8f1006d190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.589+0000 7f8f341ae700 1 --2- 172.21.15.22:0/85148317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f2c15a610 0x7f8f2c0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.589+0000 7f8f341ae700 1 -- 172.21.15.22:0/85148317 >> 172.21.15.22:0/85148317 conn(0x7f8f2c0ccb00 msgr2=0x7f8f2c15d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:01.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.589+0000 7f8f341ae700 1 -- 172.21.15.22:0/85148317 shutdown_connections 2023-07-06T01:20:01.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.589+0000 7f8f341ae700 1 -- 172.21.15.22:0/85148317 wait complete. 2023-07-06T01:20:01.604 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T01:19:22.873255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T01:19:24.019088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:20:01.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:20:01.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.731+0000 7faebfe94700 1 Processor -- start 2023-07-06T01:20:01.734 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.731+0000 7faebfe94700 1 -- start start 2023-07-06T01:20:01.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.732+0000 7faebfe94700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeb80af2a0 0x7faeb806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:01.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.732+0000 7faebfe94700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb80af670 con 0x7faeb806a000 2023-07-06T01:20:01.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.732+0000 7faebfe94700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb806d030 con 0x7faeb80cc320 2023-07-06T01:20:01.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.732+0000 7faebfe94700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb80cc7a0 con 0x7faeb80af2a0 2023-07-06T01:20:01.736 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.732+0000 7faebd42f700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7faeb806a000 0x7faeb80cbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37314/0 (socket says 172.21.15.22:37314) 2023-07-06T01:20:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.732+0000 7faebd42f700 1 -- 172.21.15.22:0/3777817917 learned_addr learned my addr 172.21.15.22:0/3777817917 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.732+0000 7faebdc30700 1 --2- 172.21.15.22:0/3777817917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeb80af2a0 0x7faeb806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.733+0000 7faebcc2e700 1 -- 172.21.15.22:0/3777817917 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416968001 0 0) 0x7faeb80af670 con 0x7faeb806a000 2023-07-06T01:20:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.733+0000 7faebcc2e700 1 -- 172.21.15.22:0/3777817917 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faea40035b0 con 0x7faeb806a000 2023-07-06T01:20:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.733+0000 7faebcc2e700 1 -- 172.21.15.22:0/3777817917 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749422586 0 0) 0x7faeb806d030 con 0x7faeb80cc320 2023-07-06T01:20:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.733+0000 7faebcc2e700 1 -- 172.21.15.22:0/3777817917 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeb80af670 con 0x7faeb80cc320 2023-07-06T01:20:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.733+0000 7faebdc30700 1 -- 172.21.15.22:0/3777817917 >> v1:172.21.15.103:6789/0 conn(0x7faeb80cc320 legacy=0x7faeb8160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:01.737 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.733+0000 7faebdc30700 1 -- 172.21.15.22:0/3777817917 >> v1:172.21.15.22:6789/0 conn(0x7faeb806a000 legacy=0x7faeb80cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:01.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.733+0000 7faebdc30700 1 -- 172.21.15.22:0/3777817917 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb80cd560 con 0x7faeb80af2a0 2023-07-06T01:20:01.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.733+0000 7faebdc30700 1 --2- 172.21.15.22:0/3777817917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeb80af2a0 0x7faeb806caf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faea8009870 tx=0x7faea8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c364e5a59cb16b41 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:01.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.733+0000 7faebcc2e700 1 -- 172.21.15.22:0/3777817917 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8011830 con 0x7faeb80af2a0 2023-07-06T01:20:01.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.733+0000 7faebcc2e700 1 -- 172.21.15.22:0/3777817917 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea8011990 con 0x7faeb80af2a0 2023-07-06T01:20:01.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.733+0000 7faebcc2e700 1 -- 172.21.15.22:0/3777817917 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8011d00 con 0x7faeb80af2a0 2023-07-06T01:20:01.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.734+0000 7faebfe94700 1 -- 172.21.15.22:0/3777817917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeb80af2a0 msgr2=0x7faeb806caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:01.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.734+0000 7faebfe94700 1 --2- 172.21.15.22:0/3777817917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeb80af2a0 0x7faeb806caf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faea8009870 tx=0x7faea8009b80 comp rx=0 tx=0).stop 2023-07-06T01:20:01.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.734+0000 7faebfe94700 1 -- 172.21.15.22:0/3777817917 shutdown_connections 2023-07-06T01:20:01.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.734+0000 7faebfe94700 1 --2- 172.21.15.22:0/3777817917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeb80af2a0 0x7faeb806caf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.734+0000 7faebfe94700 1 -- 172.21.15.22:0/3777817917 >> 172.21.15.22:0/3777817917 conn(0x7faeb80b1890 msgr2=0x7faeb80b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:01.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.734+0000 7faebfe94700 1 -- 172.21.15.22:0/3777817917 shutdown_connections 2023-07-06T01:20:01.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.734+0000 7faebfe94700 1 -- 172.21.15.22:0/3777817917 wait complete. 2023-07-06T01:20:01.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.735+0000 7faebfe94700 1 Processor -- start 2023-07-06T01:20:01.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.735+0000 7faebfe94700 1 -- start start 2023-07-06T01:20:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebfe94700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeb806a000 0x7faeb8152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebfe94700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faeb80af2a0 0x7faeb815ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebfe94700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeb80cc320 0x7faeb80cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebfe94700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb80cd850 con 0x7faeb80af2a0 2023-07-06T01:20:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebfe94700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb80cfd90 con 0x7faeb806a000 2023-07-06T01:20:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebfe94700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb80d0070 con 0x7faeb80cc320 2023-07-06T01:20:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebd42f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faeb80af2a0 0x7faeb815ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:01.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebd42f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faeb80af2a0 0x7faeb815ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46576/0 (socket says 172.21.15.22:46576) 2023-07-06T01:20:01.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebd42f700 1 -- 172.21.15.22:0/171191043 learned_addr learned my addr 172.21.15.22:0/171191043 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:01.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebdc30700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeb806a000 0x7faeb8152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:01.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebe431700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeb80cc320 0x7faeb80cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:01.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebd42f700 1 -- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeb80cc320 msgr2=0x7faeb80cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:01.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebd42f700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeb80cc320 0x7faeb80cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebd42f700 1 -- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeb806a000 msgr2=0x7faeb8152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:01.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebd42f700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeb806a000 0x7faeb8152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebd42f700 1 -- 172.21.15.22:0/171191043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea80094a0 con 0x7faeb80af2a0 2023-07-06T01:20:01.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faebd42f700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faeb80af2a0 0x7faeb815ccc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faeb400c700 tx=0x7faeb400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:01.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.736+0000 7faeae7fc700 1 -- 172.21.15.22:0/171191043 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4005d70 con 0x7faeb80af2a0 2023-07-06T01:20:01.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.737+0000 7faeae7fc700 1 -- 172.21.15.22:0/171191043 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb40070d0 con 0x7faeb80af2a0 2023-07-06T01:20:01.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.737+0000 7faebfe94700 1 -- 172.21.15.22:0/171191043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb80d0350 con 0x7faeb80af2a0 2023-07-06T01:20:01.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.737+0000 7faeae7fc700 1 -- 172.21.15.22:0/171191043 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4007460 con 0x7faeb80af2a0 2023-07-06T01:20:01.744 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.737+0000 7faebfe94700 1 -- 172.21.15.22:0/171191043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb815d200 con 0x7faeb80af2a0 2023-07-06T01:20:01.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.738+0000 7faebfe94700 1 -- 172.21.15.22:0/171191043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb815ca90 con 0x7faeb80af2a0 2023-07-06T01:20:01.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.738+0000 7faeae7fc700 1 -- 172.21.15.22:0/171191043 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faeb40075c0 con 0x7faeb80af2a0 2023-07-06T01:20:01.745 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.738+0000 7faeae7fc700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fae9c06ace0 0x7fae9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:01.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.738+0000 7faeae7fc700 1 -- 172.21.15.22:0/171191043 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faeb4088810 con 0x7faeb80af2a0 2023-07-06T01:20:01.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.739+0000 7faebdc30700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fae9c06ace0 0x7fae9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:01.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.739+0000 7faebdc30700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fae9c06ace0 0x7fae9c06d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faea80090b0 tx=0x7faea8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:01.746 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.743+0000 7faeae7fc700 1 -- 172.21.15.22:0/171191043 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faeb815ca90 con 0x7faeb80af2a0 2023-07-06T01:20:01.766 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:20:01.768 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:20:01.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.941+0000 7faebfe94700 1 -- 172.21.15.22:0/171191043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faeb815ca90 con 0x7faeb80af2a0 2023-07-06T01:20:01.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.941+0000 7faeae7fc700 1 -- 172.21.15.22:0/171191043 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7faeb815ca90 con 0x7faeb80af2a0 2023-07-06T01:20:01.945 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:20:01.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.943+0000 7faebfe94700 1 -- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fae9c06ace0 msgr2=0x7fae9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:01.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.943+0000 7faebfe94700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fae9c06ace0 0x7fae9c06d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faea80090b0 tx=0x7faea8008030 comp rx=0 tx=0).stop 2023-07-06T01:20:01.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.943+0000 7faebfe94700 1 -- 172.21.15.22:0/171191043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faeb80af2a0 msgr2=0x7faeb815ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:01.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.943+0000 7faebfe94700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faeb80af2a0 0x7faeb815ccc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faeb400c700 tx=0x7faeb400ca10 comp rx=0 tx=0).stop 2023-07-06T01:20:01.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.943+0000 7faebfe94700 1 -- 172.21.15.22:0/171191043 shutdown_connections 2023-07-06T01:20:01.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.943+0000 7faebfe94700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeb80cc320 0x7faeb80cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.943+0000 7faebfe94700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fae9c06ace0 0x7fae9c06d190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.943+0000 7faebfe94700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeb806a000 0x7faeb8152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.943+0000 7faebfe94700 1 --2- 172.21.15.22:0/171191043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faeb80af2a0 0x7faeb815ccc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.943+0000 7faebfe94700 1 -- 172.21.15.22:0/171191043 >> 172.21.15.22:0/171191043 conn(0x7faeb80b1890 msgr2=0x7faeb80c6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.943+0000 7faebfe94700 1 -- 172.21.15.22:0/171191043 shutdown_connections 2023-07-06T01:20:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:01.943+0000 7faebfe94700 1 -- 172.21.15.22:0/171191043 wait complete. 2023-07-06T01:20:01.958 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:20:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.084+0000 7f969d1cd700 1 Processor -- start 2023-07-06T01:20:02.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.084+0000 7f969d1cd700 1 -- start start 2023-07-06T01:20:02.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f969d1cd700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f969815c750 0x7f96980cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f969d1cd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f969806b820 0x7f96980cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f969d1cd700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96980d06f0 con 0x7f96980cdf80 2023-07-06T01:20:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f969d1cd700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96980ceb90 con 0x7f969806b820 2023-07-06T01:20:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f969d1cd700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96980cee70 con 0x7f969815c750 2023-07-06T01:20:02.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f969759e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f969806b820 0x7f96980cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f969659c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f96980cdf80 0x7f969806b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37326/0 (socket says 172.21.15.22:37326) 2023-07-06T01:20:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f969659c700 1 -- 172.21.15.22:0/1368310083 learned_addr learned my addr 172.21.15.22:0/1368310083 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f9696d9d700 1 --2- 172.21.15.22:0/1368310083 >> v2:172.21.15.103:3300/0 conn(0x7f969815c750 0x7f96980cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f969759e700 1 -- 172.21.15.22:0/1368310083 >> v2:172.21.15.103:3300/0 conn(0x7f969815c750 msgr2=0x7f96980cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:02.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f969759e700 1 --2- 172.21.15.22:0/1368310083 >> v2:172.21.15.103:3300/0 conn(0x7f969815c750 0x7f96980cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f969759e700 1 -- 172.21.15.22:0/1368310083 >> v1:172.21.15.22:6789/0 conn(0x7f96980cdf80 legacy=0x7f969806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f969759e700 1 -- 172.21.15.22:0/1368310083 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980cf150 con 0x7f969806b820 2023-07-06T01:20:02.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f9695d9b700 1 -- 172.21.15.22:0/1368310083 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417483712 0 0) 0x7f96980d06f0 con 0x7f96980cdf80 2023-07-06T01:20:02.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.085+0000 7f9696d9d700 1 --2- 172.21.15.22:0/1368310083 >> v2:172.21.15.103:3300/0 conn(0x7f969815c750 0x7f96980cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:02.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.086+0000 7f969759e700 1 --2- 172.21.15.22:0/1368310083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f969806b820 0x7f96980cb2e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9698070f00 tx=0x7f969000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0e7cb2dc026165 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:02.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.086+0000 7f9695d9b700 1 -- 172.21.15.22:0/1368310083 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969000e070 con 0x7f969806b820 2023-07-06T01:20:02.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.086+0000 7f9695d9b700 1 -- 172.21.15.22:0/1368310083 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969000bbd0 con 0x7f969806b820 2023-07-06T01:20:02.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.086+0000 7f9695d9b700 1 -- 172.21.15.22:0/1368310083 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690012430 con 0x7f969806b820 2023-07-06T01:20:02.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.086+0000 7f969d1cd700 1 -- 172.21.15.22:0/1368310083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f969806b820 msgr2=0x7f96980cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.086+0000 7f969d1cd700 1 --2- 172.21.15.22:0/1368310083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f969806b820 0x7f96980cb2e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9698070f00 tx=0x7f969000b330 comp rx=0 tx=0).stop 2023-07-06T01:20:02.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.086+0000 7f969d1cd700 1 -- 172.21.15.22:0/1368310083 shutdown_connections 2023-07-06T01:20:02.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.086+0000 7f969d1cd700 1 --2- 172.21.15.22:0/1368310083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f969806b820 0x7f96980cb2e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.086+0000 7f969d1cd700 1 --2- 172.21.15.22:0/1368310083 >> v2:172.21.15.103:3300/0 conn(0x7f969815c750 0x7f96980cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.086+0000 7f969d1cd700 1 -- 172.21.15.22:0/1368310083 >> 172.21.15.22:0/1368310083 conn(0x7f96980999f0 msgr2=0x7f9698099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:02.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.086+0000 7f969d1cd700 1 -- 172.21.15.22:0/1368310083 shutdown_connections 2023-07-06T01:20:02.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.086+0000 7f969d1cd700 1 -- 172.21.15.22:0/1368310083 wait complete. 2023-07-06T01:20:02.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.088+0000 7f969d1cd700 1 Processor -- start 2023-07-06T01:20:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.088+0000 7f969d1cd700 1 -- start start 2023-07-06T01:20:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969d1cd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f969806b820 0x7f96980afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969d1cd700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96980cdf80 0x7f96980b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969d1cd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f969815c750 0x7f96980b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969d1cd700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96980b4ba0 con 0x7f969815c750 2023-07-06T01:20:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969d1cd700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96980b2e00 con 0x7f969806b820 2023-07-06T01:20:02.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969d1cd700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96980b30e0 con 0x7f96980cdf80 2023-07-06T01:20:02.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969759e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f969815c750 0x7f96980b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969759e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f969815c750 0x7f96980b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46590/0 (socket says 172.21.15.22:46590) 2023-07-06T01:20:02.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969759e700 1 -- 172.21.15.22:0/3471873361 learned_addr learned my addr 172.21.15.22:0/3471873361 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:02.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969659c700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96980cdf80 0x7f96980b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f9696d9d700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f969806b820 0x7f96980afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969759e700 1 -- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96980cdf80 msgr2=0x7f96980b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969759e700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96980cdf80 0x7f96980b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969759e700 1 -- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f969806b820 msgr2=0x7f96980afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969759e700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f969806b820 0x7f96980afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969759e700 1 -- 172.21.15.22:0/3471873361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9690009a70 con 0x7f969815c750 2023-07-06T01:20:02.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969759e700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f969815c750 0x7f96980b25d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f96980b38c0 tx=0x7f9690004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:02.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f967f7fe700 1 -- 172.21.15.22:0/3471873361 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690012430 con 0x7f969815c750 2023-07-06T01:20:02.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f967f7fe700 1 -- 172.21.15.22:0/3471873361 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9690020070 con 0x7f969815c750 2023-07-06T01:20:02.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f969d1cd700 1 -- 172.21.15.22:0/3471873361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96980b33c0 con 0x7f969815c750 2023-07-06T01:20:02.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.089+0000 7f967f7fe700 1 -- 172.21.15.22:0/3471873361 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690004280 con 0x7f969815c750 2023-07-06T01:20:02.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.090+0000 7f969d1cd700 1 -- 172.21.15.22:0/3471873361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96980b2b10 con 0x7f969815c750 2023-07-06T01:20:02.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.090+0000 7f969d1cd700 1 -- 172.21.15.22:0/3471873361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96980b5030 con 0x7f969815c750 2023-07-06T01:20:02.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.092+0000 7f967f7fe700 1 -- 172.21.15.22:0/3471873361 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f96900043e0 con 0x7f969815c750 2023-07-06T01:20:02.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.092+0000 7f967f7fe700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f968406ace0 0x7f968406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.092+0000 7f967f7fe700 1 -- 172.21.15.22:0/3471873361 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9690022070 con 0x7f969815c750 2023-07-06T01:20:02.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.092+0000 7f9696d9d700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f968406ace0 0x7f968406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.093+0000 7f9696d9d700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f968406ace0 0x7f968406d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f968c006fd0 tx=0x7f968c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:02.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.095+0000 7f967f7fe700 1 -- 172.21.15.22:0/3471873361 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9690056870 con 0x7f969815c750 2023-07-06T01:20:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.296+0000 7f969d1cd700 1 -- 172.21.15.22:0/3471873361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96980b5030 con 0x7f969815c750 2023-07-06T01:20:02.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.296+0000 7f967f7fe700 1 -- 172.21.15.22:0/3471873361 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f969002c020 con 0x7f969815c750 2023-07-06T01:20:02.299 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:20:02.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.298+0000 7f969d1cd700 1 -- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f968406ace0 msgr2=0x7f968406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.298+0000 7f969d1cd700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f968406ace0 0x7f968406d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f968c006fd0 tx=0x7f968c008040 comp rx=0 tx=0).stop 2023-07-06T01:20:02.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.298+0000 7f969d1cd700 1 -- 172.21.15.22:0/3471873361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f969815c750 msgr2=0x7f96980b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.298+0000 7f969d1cd700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f969815c750 0x7f96980b25d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f96980b38c0 tx=0x7f9690004ad0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.298+0000 7f969d1cd700 1 -- 172.21.15.22:0/3471873361 shutdown_connections 2023-07-06T01:20:02.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.298+0000 7f969d1cd700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f969815c750 0x7f96980b25d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.298+0000 7f969d1cd700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f968406ace0 0x7f968406d190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.298+0000 7f969d1cd700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f969806b820 0x7f96980afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.298+0000 7f969d1cd700 1 --2- 172.21.15.22:0/3471873361 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96980cdf80 0x7f96980b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.298+0000 7f969d1cd700 1 -- 172.21.15.22:0/3471873361 >> 172.21.15.22:0/3471873361 conn(0x7f96980999f0 msgr2=0x7f96980d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.298+0000 7f969d1cd700 1 -- 172.21.15.22:0/3471873361 shutdown_connections 2023-07-06T01:20:02.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.298+0000 7f969d1cd700 1 -- 172.21.15.22:0/3471873361 wait complete. 2023-07-06T01:20:02.313 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T01:20:02.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.438+0000 7f41e19e7700 1 Processor -- start 2023-07-06T01:20:02.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.438+0000 7f41e19e7700 1 -- start start 2023-07-06T01:20:02.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.439+0000 7f41e19e7700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f41dc15cb50 0x7f41dc06a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.439+0000 7f41e19e7700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41dc0cf260 0x7f41dc160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.439+0000 7f41e19e7700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41dc0ca2b0 con 0x7f41dc06a540 2023-07-06T01:20:02.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.439+0000 7f41e19e7700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41dc0af440 con 0x7f41dc15cb50 2023-07-06T01:20:02.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.439+0000 7f41e19e7700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc06a970 con 0x7f41dc0cf260 2023-07-06T01:20:02.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.439+0000 7f41daffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f41dc15cb50 0x7f41dc06a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.439+0000 7f41daffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f41dc15cb50 0x7f41dc06a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46594/0 (socket says 172.21.15.22:46594) 2023-07-06T01:20:02.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.439+0000 7f41da7fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f41dc06a540 0x7f41dc0ceb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37338/0 (socket says 172.21.15.22:37338) 2023-07-06T01:20:02.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.439+0000 7f41daffd700 1 -- 172.21.15.22:0/3250560463 learned_addr learned my addr 172.21.15.22:0/3250560463 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:02.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.440+0000 7f41db7fe700 1 --2- 172.21.15.22:0/3250560463 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41dc0cf260 0x7f41dc160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.440+0000 7f41daffd700 1 -- 172.21.15.22:0/3250560463 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41dc0cf260 msgr2=0x7f41dc160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:02.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.440+0000 7f41daffd700 1 --2- 172.21.15.22:0/3250560463 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41dc0cf260 0x7f41dc160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.440+0000 7f41daffd700 1 -- 172.21.15.22:0/3250560463 >> v1:172.21.15.22:6789/0 conn(0x7f41dc06a540 legacy=0x7f41dc0ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.440+0000 7f41d9ffb700 1 -- 172.21.15.22:0/3250560463 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 85926237 0 0) 0x7f41dc0ca2b0 con 0x7f41dc06a540 2023-07-06T01:20:02.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.440+0000 7f41daffd700 1 -- 172.21.15.22:0/3250560463 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0cfc90 con 0x7f41dc15cb50 2023-07-06T01:20:02.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.440+0000 7f41daffd700 1 --2- 172.21.15.22:0/3250560463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41dc15cb50 0x7f41dc06a000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f41d00098f0 tx=0x7f41d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efc2d4d401484cf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:02.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.440+0000 7f41d9ffb700 1 -- 172.21.15.22:0/3250560463 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000b8a0 con 0x7f41dc15cb50 2023-07-06T01:20:02.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.440+0000 7f41d9ffb700 1 -- 172.21.15.22:0/3250560463 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d000ba00 con 0x7f41dc15cb50 2023-07-06T01:20:02.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.440+0000 7f41d9ffb700 1 -- 172.21.15.22:0/3250560463 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000bd90 con 0x7f41dc15cb50 2023-07-06T01:20:02.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.441+0000 7f41e19e7700 1 -- 172.21.15.22:0/3250560463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41dc15cb50 msgr2=0x7f41dc06a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.441+0000 7f41e19e7700 1 --2- 172.21.15.22:0/3250560463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41dc15cb50 0x7f41dc06a000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f41d00098f0 tx=0x7f41d0009c00 comp rx=0 tx=0).stop 2023-07-06T01:20:02.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.441+0000 7f41e19e7700 1 -- 172.21.15.22:0/3250560463 shutdown_connections 2023-07-06T01:20:02.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.441+0000 7f41e19e7700 1 --2- 172.21.15.22:0/3250560463 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41dc0cf260 0x7f41dc160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.441+0000 7f41e19e7700 1 --2- 172.21.15.22:0/3250560463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41dc15cb50 0x7f41dc06a000 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.441+0000 7f41e19e7700 1 -- 172.21.15.22:0/3250560463 >> 172.21.15.22:0/3250560463 conn(0x7f41dc06b110 msgr2=0x7f41dc06b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:02.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.441+0000 7f41e19e7700 1 -- 172.21.15.22:0/3250560463 shutdown_connections 2023-07-06T01:20:02.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.441+0000 7f41e19e7700 1 -- 172.21.15.22:0/3250560463 wait complete. 2023-07-06T01:20:02.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.442+0000 7f41e19e7700 1 Processor -- start 2023-07-06T01:20:02.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.442+0000 7f41e19e7700 1 -- start start 2023-07-06T01:20:02.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.443+0000 7f41e19e7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41dc06a540 0x7f41dc15b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.443+0000 7f41e19e7700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41dc0cf260 0x7f41dc0ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.443+0000 7f41e19e7700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41dc15cb50 0x7f41dc1460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.443+0000 7f41e19e7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc15ba70 con 0x7f41dc06a540 2023-07-06T01:20:02.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.443+0000 7f41e19e7700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc0ad7c0 con 0x7f41dc15cb50 2023-07-06T01:20:02.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.443+0000 7f41e19e7700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc0ad9d0 con 0x7f41dc0cf260 2023-07-06T01:20:02.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.443+0000 7f41daffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41dc06a540 0x7f41dc15b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41daffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41dc06a540 0x7f41dc15b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46600/0 (socket says 172.21.15.22:46600) 2023-07-06T01:20:02.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41daffd700 1 -- 172.21.15.22:0/1391339283 learned_addr learned my addr 172.21.15.22:0/1391339283 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:02.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41da7fc700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41dc0cf260 0x7f41dc0ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41db7fe700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41dc15cb50 0x7f41dc1460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41daffd700 1 -- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41dc0cf260 msgr2=0x7f41dc0ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41daffd700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41dc0cf260 0x7f41dc0ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41daffd700 1 -- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41dc15cb50 msgr2=0x7f41dc1460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41daffd700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41dc15cb50 0x7f41dc1460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41daffd700 1 -- 172.21.15.22:0/1391339283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d0009550 con 0x7f41dc06a540 2023-07-06T01:20:02.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41daffd700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41dc06a540 0x7f41dc15b530 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009c40 tx=0x7f41d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:02.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41c3fff700 1 -- 172.21.15.22:0/1391339283 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d00036a0 con 0x7f41dc06a540 2023-07-06T01:20:02.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41c3fff700 1 -- 172.21.15.22:0/1391339283 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d001e070 con 0x7f41dc06a540 2023-07-06T01:20:02.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41e19e7700 1 -- 172.21.15.22:0/1391339283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41dc0adcb0 con 0x7f41dc06a540 2023-07-06T01:20:02.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41c3fff700 1 -- 172.21.15.22:0/1391339283 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0003a20 con 0x7f41dc06a540 2023-07-06T01:20:02.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.444+0000 7f41e19e7700 1 -- 172.21.15.22:0/1391339283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41dc0ae270 con 0x7f41dc06a540 2023-07-06T01:20:02.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.445+0000 7f41e19e7700 1 -- 172.21.15.22:0/1391339283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41dc066910 con 0x7f41dc06a540 2023-07-06T01:20:02.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.446+0000 7f41c3fff700 1 -- 172.21.15.22:0/1391339283 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41d0003b80 con 0x7f41dc06a540 2023-07-06T01:20:02.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.446+0000 7f41c3fff700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f41c806ace0 0x7f41c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.446+0000 7f41c3fff700 1 -- 172.21.15.22:0/1391339283 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f41d0020070 con 0x7f41dc06a540 2023-07-06T01:20:02.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.446+0000 7f41da7fc700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f41c806ace0 0x7f41c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.447+0000 7f41da7fc700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f41c806ace0 0x7f41c806d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f41dc073630 tx=0x7f41cc007be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:02.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.450+0000 7f41c3fff700 1 -- 172.21.15.22:0/1391339283 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f41d00559c0 con 0x7f41dc06a540 2023-07-06T01:20:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.645+0000 7f41e19e7700 1 -- 172.21.15.22:0/1391339283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f41dc0744b0 con 0x7f41dc06a540 2023-07-06T01:20:02.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.646+0000 7f41c3fff700 1 -- 172.21.15.22:0/1391339283 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f41d0058fe0 con 0x7f41dc06a540 2023-07-06T01:20:02.649 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-07-06T01:20:02.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.648+0000 7f41e19e7700 1 -- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f41c806ace0 msgr2=0x7f41c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.648+0000 7f41e19e7700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f41c806ace0 0x7f41c806d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f41dc073630 tx=0x7f41cc007be0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.648+0000 7f41e19e7700 1 -- 172.21.15.22:0/1391339283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41dc06a540 msgr2=0x7f41dc15b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.648+0000 7f41e19e7700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41dc06a540 0x7f41dc15b530 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009c40 tx=0x7f41d0004750 comp rx=0 tx=0).stop 2023-07-06T01:20:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.648+0000 7f41e19e7700 1 -- 172.21.15.22:0/1391339283 shutdown_connections 2023-07-06T01:20:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.648+0000 7f41e19e7700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f41c806ace0 0x7f41c806d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.648+0000 7f41e19e7700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41dc15cb50 0x7f41dc1460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.648+0000 7f41e19e7700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41dc06a540 0x7f41dc15b530 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.648+0000 7f41e19e7700 1 --2- 172.21.15.22:0/1391339283 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41dc0cf260 0x7f41dc0ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.648+0000 7f41e19e7700 1 -- 172.21.15.22:0/1391339283 >> 172.21.15.22:0/1391339283 conn(0x7f41dc06b110 msgr2=0x7f41dc1e7830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:02.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.648+0000 7f41e19e7700 1 -- 172.21.15.22:0/1391339283 shutdown_connections 2023-07-06T01:20:02.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.648+0000 7f41e19e7700 1 -- 172.21.15.22:0/1391339283 wait complete. 2023-07-06T01:20:02.663 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T01:20:02.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.787+0000 7f31979ae700 1 Processor -- start 2023-07-06T01:20:02.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.788+0000 7f31979ae700 1 -- start start 2023-07-06T01:20:02.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f31979ae700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3190159870 0x7f319015df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f31979ae700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f31900715f0 0x7f31900cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f31979ae700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31900ce550 con 0x7f31900d0130 2023-07-06T01:20:02.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f31979ae700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31900d05b0 con 0x7f31900715f0 2023-07-06T01:20:02.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f31979ae700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31900cdb60 con 0x7f3190159870 2023-07-06T01:20:02.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f3194f49700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f31900715f0 0x7f31900cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f3194f49700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f31900715f0 0x7f31900cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46608/0 (socket says 172.21.15.22:46608) 2023-07-06T01:20:02.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f3194f49700 1 -- 172.21.15.22:0/657971280 learned_addr learned my addr 172.21.15.22:0/657971280 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:02.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f319574a700 1 --2- 172.21.15.22:0/657971280 >> v2:172.21.15.103:3300/0 conn(0x7f3190159870 0x7f319015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f3194f49700 1 -- 172.21.15.22:0/657971280 >> v2:172.21.15.103:3300/0 conn(0x7f3190159870 msgr2=0x7f319015df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:02.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f3194f49700 1 --2- 172.21.15.22:0/657971280 >> v2:172.21.15.103:3300/0 conn(0x7f3190159870 0x7f319015df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f3194f49700 1 -- 172.21.15.22:0/657971280 >> v1:172.21.15.103:6789/0 conn(0x7f31900d0130 legacy=0x7f31900cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f3194f49700 1 -- 172.21.15.22:0/657971280 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319015d490 con 0x7f31900715f0 2023-07-06T01:20:02.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.789+0000 7f3194f49700 1 --2- 172.21.15.22:0/657971280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31900715f0 0x7f31900cfbf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3190066c70 tx=0x7f318c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e3fe2592f81547 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:02.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.790+0000 7f3187fff700 1 -- 172.21.15.22:0/657971280 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c00dd30 con 0x7f31900715f0 2023-07-06T01:20:02.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.790+0000 7f3187fff700 1 -- 172.21.15.22:0/657971280 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c00de90 con 0x7f31900715f0 2023-07-06T01:20:02.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.790+0000 7f3187fff700 1 -- 172.21.15.22:0/657971280 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c010690 con 0x7f31900715f0 2023-07-06T01:20:02.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.790+0000 7f31979ae700 1 -- 172.21.15.22:0/657971280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31900715f0 msgr2=0x7f31900cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.790+0000 7f31979ae700 1 --2- 172.21.15.22:0/657971280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31900715f0 0x7f31900cfbf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3190066c70 tx=0x7f318c00be70 comp rx=0 tx=0).stop 2023-07-06T01:20:02.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.790+0000 7f31979ae700 1 -- 172.21.15.22:0/657971280 shutdown_connections 2023-07-06T01:20:02.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.790+0000 7f31979ae700 1 --2- 172.21.15.22:0/657971280 >> v2:172.21.15.103:3300/0 conn(0x7f3190159870 0x7f319015df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.790+0000 7f31979ae700 1 --2- 172.21.15.22:0/657971280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31900715f0 0x7f31900cfbf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.790+0000 7f31979ae700 1 -- 172.21.15.22:0/657971280 >> 172.21.15.22:0/657971280 conn(0x7f319006b110 msgr2=0x7f319006b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:02.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.790+0000 7f31979ae700 1 -- 172.21.15.22:0/657971280 shutdown_connections 2023-07-06T01:20:02.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.790+0000 7f31979ae700 1 -- 172.21.15.22:0/657971280 wait complete. 2023-07-06T01:20:02.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.792+0000 7f31979ae700 1 Processor -- start 2023-07-06T01:20:02.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.792+0000 7f31979ae700 1 -- start start 2023-07-06T01:20:02.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f31979ae700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31900715f0 0x7f319014fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f31979ae700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31900d0130 0x7f31900b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f31979ae700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3190159870 0x7f31901532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f31979ae700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3190150310 con 0x7f3190159870 2023-07-06T01:20:02.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f31979ae700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31900b4470 con 0x7f31900d0130 2023-07-06T01:20:02.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f31979ae700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31900b4750 con 0x7f31900715f0 2023-07-06T01:20:02.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f3195f4b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3190159870 0x7f31901532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f3195f4b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3190159870 0x7f31901532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46618/0 (socket says 172.21.15.22:46618) 2023-07-06T01:20:02.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f3195f4b700 1 -- 172.21.15.22:0/3142681689 learned_addr learned my addr 172.21.15.22:0/3142681689 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:02.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f3194f49700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31900d0130 0x7f31900b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f319574a700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31900715f0 0x7f319014fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f3195f4b700 1 -- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31900715f0 msgr2=0x7f319014fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f3195f4b700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31900715f0 0x7f319014fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f3195f4b700 1 -- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31900d0130 msgr2=0x7f31900b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:02.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f3195f4b700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31900d0130 0x7f31900b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:02.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f3195f4b700 1 -- 172.21.15.22:0/3142681689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c00b7f0 con 0x7f3190159870 2023-07-06T01:20:02.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f3194f49700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31900d0130 0x7f31900b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:02.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.793+0000 7f3195f4b700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3190159870 0x7f31901532e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f318800cc50 tx=0x7f318800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:02.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.794+0000 7f3185ffb700 1 -- 172.21.15.22:0/3142681689 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318800ad80 con 0x7f3190159870 2023-07-06T01:20:02.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.794+0000 7f3185ffb700 1 -- 172.21.15.22:0/3142681689 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318800ce50 con 0x7f3190159870 2023-07-06T01:20:02.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.794+0000 7f3185ffb700 1 -- 172.21.15.22:0/3142681689 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188003c40 con 0x7f3190159870 2023-07-06T01:20:02.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.794+0000 7f31979ae700 1 -- 172.21.15.22:0/3142681689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31900b4a30 con 0x7f3190159870 2023-07-06T01:20:02.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.794+0000 7f31979ae700 1 -- 172.21.15.22:0/3142681689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31900b4ff0 con 0x7f3190159870 2023-07-06T01:20:02.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.795+0000 7f31979ae700 1 -- 172.21.15.22:0/3142681689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31900744b0 con 0x7f3190159870 2023-07-06T01:20:02.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.795+0000 7f3185ffb700 1 -- 172.21.15.22:0/3142681689 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f31880050d0 con 0x7f3190159870 2023-07-06T01:20:02.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.795+0000 7f3185ffb700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f317406ace0 0x7f317406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:02.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.796+0000 7f3185ffb700 1 -- 172.21.15.22:0/3142681689 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3188089630 con 0x7f3190159870 2023-07-06T01:20:02.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.796+0000 7f319574a700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f317406ace0 0x7f317406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:02.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.797+0000 7f319574a700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f317406ace0 0x7f317406d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3180009770 tx=0x7f3180008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:02.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.800+0000 7f3185ffb700 1 -- 172.21.15.22:0/3142681689 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3188054640 con 0x7f3190159870 2023-07-06T01:20:02.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.996+0000 7f31979ae700 1 -- 172.21.15.22:0/3142681689 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f319006d010 con 0x7f3190159870 2023-07-06T01:20:03.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.997+0000 7f3185ffb700 1 -- 172.21.15.22:0/3142681689 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3188057c60 con 0x7f3190159870 2023-07-06T01:20:03.000 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-07-06T01:20:03.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.999+0000 7f31979ae700 1 -- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f317406ace0 msgr2=0x7f317406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:03.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.999+0000 7f31979ae700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f317406ace0 0x7f317406d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3180009770 tx=0x7f3180008040 comp rx=0 tx=0).stop 2023-07-06T01:20:03.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.999+0000 7f31979ae700 1 -- 172.21.15.22:0/3142681689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3190159870 msgr2=0x7f31901532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:03.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.999+0000 7f31979ae700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3190159870 0x7f31901532e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f318800cc50 tx=0x7f318800a340 comp rx=0 tx=0).stop 2023-07-06T01:20:03.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.999+0000 7f31979ae700 1 -- 172.21.15.22:0/3142681689 shutdown_connections 2023-07-06T01:20:03.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.999+0000 7f31979ae700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3190159870 0x7f31901532e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.999+0000 7f31979ae700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31900715f0 0x7f319014fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.999+0000 7f31979ae700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f317406ace0 0x7f317406d190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.999+0000 7f31979ae700 1 --2- 172.21.15.22:0/3142681689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31900d0130 0x7f31900b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.999+0000 7f31979ae700 1 -- 172.21.15.22:0/3142681689 >> 172.21.15.22:0/3142681689 conn(0x7f319006b110 msgr2=0x7f319006b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:03.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.999+0000 7f31979ae700 1 -- 172.21.15.22:0/3142681689 shutdown_connections 2023-07-06T01:20:03.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:02.999+0000 7f31979ae700 1 -- 172.21.15.22:0/3142681689 wait complete. 2023-07-06T01:20:03.014 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T01:20:03.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.139+0000 7f8a64161700 1 Processor -- start 2023-07-06T01:20:03.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.139+0000 7f8a64161700 1 -- start start 2023-07-06T01:20:03.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.140+0000 7f8a64161700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a5c06a710 0x7f8a5c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:03.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.140+0000 7f8a64161700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f8a5c0cbc10 0x7f8a5c0cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:03.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.140+0000 7f8a64161700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5c0af670 con 0x7f8a5c0af2a0 2023-07-06T01:20:03.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.140+0000 7f8a64161700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5c0cd560 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.140+0000 7f8a64161700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c15cab0 con 0x7f8a5c06a710 2023-07-06T01:20:03.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.140+0000 7f8a626fe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f8a5c0cbc10 0x7f8a5c0cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:03.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.140+0000 7f8a616fc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a5c06a710 0x7f8a5c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:03.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.140+0000 7f8a626fe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f8a5c0cbc10 0x7f8a5c0cc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:35350/0 (socket says 172.21.15.22:35350) 2023-07-06T01:20:03.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.140+0000 7f8a616fc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a5c06a710 0x7f8a5c06caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:57762/0 (socket says 172.21.15.22:57762) 2023-07-06T01:20:03.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.140+0000 7f8a61efd700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f8a5c0af2a0 0x7f8a5c06a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:39430/0 (socket says 172.21.15.22:39430) 2023-07-06T01:20:03.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.140+0000 7f8a626fe700 1 -- 172.21.15.22:0/618131382 learned_addr learned my addr 172.21.15.22:0/618131382 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:03.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.141+0000 7f8a616fc700 1 -- 172.21.15.22:0/618131382 >> v2:172.21.15.103:3300/0 conn(0x7f8a5c0cbc10 msgr2=0x7f8a5c0cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:03.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.141+0000 7f8a616fc700 1 --2- 172.21.15.22:0/618131382 >> v2:172.21.15.103:3300/0 conn(0x7f8a5c0cbc10 0x7f8a5c0cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.141+0000 7f8a616fc700 1 -- 172.21.15.22:0/618131382 >> v1:172.21.15.103:6789/0 conn(0x7f8a5c0af2a0 legacy=0x7f8a5c06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:03.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.141+0000 7f8a616fc700 1 -- 172.21.15.22:0/618131382 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c06d030 con 0x7f8a5c06a710 2023-07-06T01:20:03.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.141+0000 7f8a626fe700 1 --2- 172.21.15.22:0/618131382 >> v2:172.21.15.103:3300/0 conn(0x7f8a5c0cbc10 0x7f8a5c0cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:03.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.141+0000 7f8a616fc700 1 --2- 172.21.15.22:0/618131382 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a5c06a710 0x7f8a5c06caf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009840 tx=0x7f8a58009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ea7015d51f99e9f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:03.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.141+0000 7f8a60efb700 1 -- 172.21.15.22:0/618131382 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58011830 con 0x7f8a5c06a710 2023-07-06T01:20:03.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.141+0000 7f8a60efb700 1 -- 172.21.15.22:0/618131382 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a58011990 con 0x7f8a5c06a710 2023-07-06T01:20:03.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.141+0000 7f8a60efb700 1 -- 172.21.15.22:0/618131382 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58011d20 con 0x7f8a5c06a710 2023-07-06T01:20:03.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.141+0000 7f8a64161700 1 -- 172.21.15.22:0/618131382 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a5c06a710 msgr2=0x7f8a5c06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:03.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.141+0000 7f8a64161700 1 --2- 172.21.15.22:0/618131382 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a5c06a710 0x7f8a5c06caf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009840 tx=0x7f8a58009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:03.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.142+0000 7f8a64161700 1 -- 172.21.15.22:0/618131382 shutdown_connections 2023-07-06T01:20:03.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.142+0000 7f8a64161700 1 --2- 172.21.15.22:0/618131382 >> v2:172.21.15.103:3300/0 conn(0x7f8a5c0cbc10 0x7f8a5c0cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.142+0000 7f8a64161700 1 --2- 172.21.15.22:0/618131382 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a5c06a710 0x7f8a5c06caf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.142+0000 7f8a64161700 1 -- 172.21.15.22:0/618131382 >> 172.21.15.22:0/618131382 conn(0x7f8a5c0b1890 msgr2=0x7f8a5c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:03.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.142+0000 7f8a64161700 1 -- 172.21.15.22:0/618131382 shutdown_connections 2023-07-06T01:20:03.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.142+0000 7f8a64161700 1 -- 172.21.15.22:0/618131382 wait complete. 2023-07-06T01:20:03.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.143+0000 7f8a64161700 1 Processor -- start 2023-07-06T01:20:03.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.143+0000 7f8a64161700 1 -- start start 2023-07-06T01:20:03.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a64161700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a5c06a710 0x7f8a5c147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:03.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a64161700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a5c0af2a0 0x7f8a5c1467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:03.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a64161700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a5c0cbc10 0x7f8a5c146d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:03.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a64161700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c14a500 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a64161700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c147240 con 0x7f8a5c0af2a0 2023-07-06T01:20:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a64161700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c0c85b0 con 0x7f8a5c06a710 2023-07-06T01:20:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a61efd700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a5c06a710 0x7f8a5c147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a626fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a5c0cbc10 0x7f8a5c146d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a616fc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a5c0af2a0 0x7f8a5c1467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a626fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a5c0cbc10 0x7f8a5c146d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46632/0 (socket says 172.21.15.22:46632) 2023-07-06T01:20:03.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a616fc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a5c0af2a0 0x7f8a5c1467c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:35354/0 (socket says 172.21.15.22:35354) 2023-07-06T01:20:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a626fe700 1 -- 172.21.15.22:0/1119567471 learned_addr learned my addr 172.21.15.22:0/1119567471 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a61efd700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a5c06a710 0x7f8a5c147480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:57766/0 (socket says 172.21.15.22:57766) 2023-07-06T01:20:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a626fe700 1 -- 172.21.15.22:0/1119567471 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a5c06a710 msgr2=0x7f8a5c147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a626fe700 1 --2- 172.21.15.22:0/1119567471 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a5c06a710 0x7f8a5c147480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a626fe700 1 -- 172.21.15.22:0/1119567471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a5c0af2a0 msgr2=0x7f8a5c1467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a626fe700 1 --2- 172.21.15.22:0/1119567471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a5c0af2a0 0x7f8a5c1467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a626fe700 1 -- 172.21.15.22:0/1119567471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a54009b30 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a626fe700 1 --2- 172.21.15.22:0/1119567471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a5c0cbc10 0x7f8a5c146d00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400efc0 tx=0x7f8a5400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:03.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a527fc700 1 -- 172.21.15.22:0/1119567471 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54018070 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.144+0000 7f8a527fc700 1 -- 172.21.15.22:0/1119567471 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5400f040 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.145+0000 7f8a527fc700 1 -- 172.21.15.22:0/1119567471 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54003c40 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.145+0000 7f8a64161700 1 -- 172.21.15.22:0/1119567471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a580094a0 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.145+0000 7f8a64161700 1 -- 172.21.15.22:0/1119567471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5c14a220 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.145+0000 7f8a64161700 1 -- 172.21.15.22:0/1119567471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5c0744b0 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.146+0000 7f8a527fc700 1 -- 172.21.15.22:0/1119567471 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8a5400c880 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.146+0000 7f8a527fc700 1 --2- 172.21.15.22:0/1119567471 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8a4006ace0 0x7f8a4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:03.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.146+0000 7f8a527fc700 1 -- 172.21.15.22:0/1119567471 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8a54089490 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.146+0000 7f8a61efd700 1 --2- 172.21.15.22:0/1119567471 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8a4006ace0 0x7f8a4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:03.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.147+0000 7f8a61efd700 1 --2- 172.21.15.22:0/1119567471 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8a4006ace0 0x7f8a4006d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c000f60 tx=0x7f8a4c007f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:03.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.151+0000 7f8a527fc700 1 -- 172.21.15.22:0/1119567471 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8a54058060 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.364+0000 7f8a64161700 1 -- 172.21.15.22:0/1119567471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8a5c0c86f0 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.365+0000 7f8a527fc700 1 -- 172.21.15.22:0/1119567471 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8a54009ce0 con 0x7f8a5c0cbc10 2023-07-06T01:20:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.367+0000 7f8a64161700 1 -- 172.21.15.22:0/1119567471 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8a4006ace0 msgr2=0x7f8a4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.367+0000 7f8a64161700 1 --2- 172.21.15.22:0/1119567471 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8a4006ace0 0x7f8a4006d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c000f60 tx=0x7f8a4c007f10 comp rx=0 tx=0).stop 2023-07-06T01:20:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.367+0000 7f8a64161700 1 -- 172.21.15.22:0/1119567471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a5c0cbc10 msgr2=0x7f8a5c146d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.367+0000 7f8a64161700 1 --2- 172.21.15.22:0/1119567471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a5c0cbc10 0x7f8a5c146d00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400efc0 tx=0x7f8a5400c3a0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.367+0000 7f8a64161700 1 -- 172.21.15.22:0/1119567471 shutdown_connections 2023-07-06T01:20:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.367+0000 7f8a64161700 1 --2- 172.21.15.22:0/1119567471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a5c0cbc10 0x7f8a5c146d00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.367+0000 7f8a64161700 1 --2- 172.21.15.22:0/1119567471 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a5c06a710 0x7f8a5c147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.367+0000 7f8a64161700 1 --2- 172.21.15.22:0/1119567471 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8a4006ace0 0x7f8a4006d190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.367+0000 7f8a64161700 1 --2- 172.21.15.22:0/1119567471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a5c0af2a0 0x7f8a5c1467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.367+0000 7f8a64161700 1 -- 172.21.15.22:0/1119567471 >> 172.21.15.22:0/1119567471 conn(0x7f8a5c0b1890 msgr2=0x7f8a5c15c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.367+0000 7f8a64161700 1 -- 172.21.15.22:0/1119567471 shutdown_connections 2023-07-06T01:20:03.371 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.367+0000 7f8a64161700 1 -- 172.21.15.22:0/1119567471 wait complete. 2023-07-06T01:20:03.372 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-07-06T01:20:03.382 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-07-06T01:20:03.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:03.383 DEBUG:teuthology.orchestra.run.smithi103:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-07-06T01:20:03.425 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi103.front.sepia.ceph.com 2023-07-06T01:20:03.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:03.428 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T01:20:03.493 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:20:03.493 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:20:03.511 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:20:03.512 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:20:03.569 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:20:03.569 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:20:03.628 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:20:03.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.789+0000 7fbaf737c700 1 Processor -- start 2023-07-06T01:20:03.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.789+0000 7fbaf737c700 1 -- start start 2023-07-06T01:20:03.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.790+0000 7fbaf737c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbaf0159870 0x7fbaf015df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:03.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.790+0000 7fbaf737c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbaf00d0300 0x7fbaf00ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:03.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.790+0000 7fbaf737c700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf00ce550 con 0x7fbaf00715f0 2023-07-06T01:20:03.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.790+0000 7fbaf737c700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf00cdb60 con 0x7fbaf0159870 2023-07-06T01:20:03.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.790+0000 7fbaf737c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf015d490 con 0x7fbaf00d0300 2023-07-06T01:20:03.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.790+0000 7fbaf5118700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbaf0159870 0x7fbaf015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:03.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.790+0000 7fbaf4917700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fbaf00715f0 0x7fbaf00cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37380/0 (socket says 172.21.15.22:37380) 2023-07-06T01:20:03.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.790+0000 7fbaf4917700 1 -- 172.21.15.22:0/1785087018 learned_addr learned my addr 172.21.15.22:0/1785087018 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:03.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbaf5118700 1 -- 172.21.15.22:0/1785087018 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbaf00d0300 msgr2=0x7fbaf00ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:03.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbaf5919700 1 --2- 172.21.15.22:0/1785087018 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbaf00d0300 0x7fbaf00ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:03.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbaf5118700 1 --2- 172.21.15.22:0/1785087018 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbaf00d0300 0x7fbaf00ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbaf5118700 1 -- 172.21.15.22:0/1785087018 >> v1:172.21.15.22:6789/0 conn(0x7fbaf00715f0 legacy=0x7fbaf00cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:03.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbae7fff700 1 -- 172.21.15.22:0/1785087018 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082838334 0 0) 0x7fbaf00ce550 con 0x7fbaf00715f0 2023-07-06T01:20:03.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbaf5118700 1 -- 172.21.15.22:0/1785087018 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf015d770 con 0x7fbaf0159870 2023-07-06T01:20:03.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbaf5919700 1 --2- 172.21.15.22:0/1785087018 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbaf00d0300 0x7fbaf00ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:03.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbaf5118700 1 --2- 172.21.15.22:0/1785087018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaf0159870 0x7fbaf015df70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009840 tx=0x7fbae0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc5687aeb897f78 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:03.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbae7fff700 1 -- 172.21.15.22:0/1785087018 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0011830 con 0x7fbaf0159870 2023-07-06T01:20:03.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbae7fff700 1 -- 172.21.15.22:0/1785087018 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae0011990 con 0x7fbaf0159870 2023-07-06T01:20:03.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbae7fff700 1 -- 172.21.15.22:0/1785087018 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0011d20 con 0x7fbaf0159870 2023-07-06T01:20:03.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbaf737c700 1 -- 172.21.15.22:0/1785087018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaf0159870 msgr2=0x7fbaf015df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:03.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbaf737c700 1 --2- 172.21.15.22:0/1785087018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaf0159870 0x7fbaf015df70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009840 tx=0x7fbae0009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:03.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbaf737c700 1 -- 172.21.15.22:0/1785087018 shutdown_connections 2023-07-06T01:20:03.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbaf737c700 1 --2- 172.21.15.22:0/1785087018 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbaf00d0300 0x7fbaf00ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbaf737c700 1 --2- 172.21.15.22:0/1785087018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaf0159870 0x7fbaf015df70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.791+0000 7fbaf737c700 1 -- 172.21.15.22:0/1785087018 >> 172.21.15.22:0/1785087018 conn(0x7fbaf006b110 msgr2=0x7fbaf006b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:03.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.792+0000 7fbaf737c700 1 -- 172.21.15.22:0/1785087018 shutdown_connections 2023-07-06T01:20:03.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.792+0000 7fbaf737c700 1 -- 172.21.15.22:0/1785087018 wait complete. 2023-07-06T01:20:03.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.793+0000 7fbaf737c700 1 Processor -- start 2023-07-06T01:20:03.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.793+0000 7fbaf737c700 1 -- start start 2023-07-06T01:20:03.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.793+0000 7fbaf737c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbaf00715f0 0x7fbaf00c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:03.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf737c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaf00d0300 0x7fbaf0156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:03.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf737c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaf0159870 0x7fbaf00c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:03.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf737c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf00c8890 con 0x7fbaf0159870 2023-07-06T01:20:03.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf737c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf00c72b0 con 0x7fbaf00d0300 2023-07-06T01:20:03.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf737c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf00c7590 con 0x7fbaf00715f0 2023-07-06T01:20:03.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf5919700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaf0159870 0x7fbaf00c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:03.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf5919700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaf0159870 0x7fbaf00c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46646/0 (socket says 172.21.15.22:46646) 2023-07-06T01:20:03.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf5919700 1 -- 172.21.15.22:0/3921322017 learned_addr learned my addr 172.21.15.22:0/3921322017 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:03.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf5118700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbaf00715f0 0x7fbaf00c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:03.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf4917700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaf00d0300 0x7fbaf0156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:03.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf5919700 1 -- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbaf00715f0 msgr2=0x7fbaf00c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:03.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf5919700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbaf00715f0 0x7fbaf00c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf5919700 1 -- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaf00d0300 msgr2=0x7fbaf0156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:03.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf5919700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaf00d0300 0x7fbaf0156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:03.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf5919700 1 -- 172.21.15.22:0/3921322017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae00094a0 con 0x7fbaf0159870 2023-07-06T01:20:03.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf5118700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbaf00715f0 0x7fbaf00c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:03.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf4917700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaf00d0300 0x7fbaf0156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:03.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf5919700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaf0159870 0x7fbaf00c58c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbae800cef0 tx=0x7fbae800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:03.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbae5ffb700 1 -- 172.21.15.22:0/3921322017 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae800ae50 con 0x7fbaf0159870 2023-07-06T01:20:03.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbae5ffb700 1 -- 172.21.15.22:0/3921322017 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae8015440 con 0x7fbaf0159870 2023-07-06T01:20:03.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbae5ffb700 1 -- 172.21.15.22:0/3921322017 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae80157f0 con 0x7fbaf0159870 2023-07-06T01:20:03.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.794+0000 7fbaf737c700 1 -- 172.21.15.22:0/3921322017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf00c7870 con 0x7fbaf0159870 2023-07-06T01:20:03.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.795+0000 7fbaf737c700 1 -- 172.21.15.22:0/3921322017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf00c7e30 con 0x7fbaf0159870 2023-07-06T01:20:03.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.796+0000 7fbaf737c700 1 -- 172.21.15.22:0/3921322017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf00744b0 con 0x7fbaf0159870 2023-07-06T01:20:03.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.796+0000 7fbae5ffb700 1 -- 172.21.15.22:0/3921322017 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbae800d430 con 0x7fbaf0159870 2023-07-06T01:20:03.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.796+0000 7fbae5ffb700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbad406ace0 0x7fbad406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:03.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.796+0000 7fbae5ffb700 1 -- 172.21.15.22:0/3921322017 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbae8089a10 con 0x7fbaf0159870 2023-07-06T01:20:03.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.797+0000 7fbaf5118700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbad406ace0 0x7fbad406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:03.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.797+0000 7fbaf5118700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbad406ace0 0x7fbad406d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbaf01540b0 tx=0x7fbae0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:03.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.801+0000 7fbae5ffb700 1 -- 172.21.15.22:0/3921322017 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbae8054b50 con 0x7fbaf0159870 2023-07-06T01:20:04.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.997+0000 7fbaf737c700 1 -- 172.21.15.22:0/3921322017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbaf006d010 con 0x7fbaf0159870 2023-07-06T01:20:04.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:03.998+0000 7fbae5ffb700 1 -- 172.21.15.22:0/3921322017 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7fbae801a090 con 0x7fbaf0159870 2023-07-06T01:20:04.003 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:04.003 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:20:04.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.000+0000 7fbaf737c700 1 -- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbad406ace0 msgr2=0x7fbad406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.000+0000 7fbaf737c700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbad406ace0 0x7fbad406d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbaf01540b0 tx=0x7fbae0008030 comp rx=0 tx=0).stop 2023-07-06T01:20:04.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.000+0000 7fbaf737c700 1 -- 172.21.15.22:0/3921322017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaf0159870 msgr2=0x7fbaf00c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.000+0000 7fbaf737c700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaf0159870 0x7fbaf00c58c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbae800cef0 tx=0x7fbae800a340 comp rx=0 tx=0).stop 2023-07-06T01:20:04.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.000+0000 7fbaf737c700 1 -- 172.21.15.22:0/3921322017 shutdown_connections 2023-07-06T01:20:04.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.000+0000 7fbaf737c700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbaf0159870 0x7fbaf00c58c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.000+0000 7fbaf737c700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbaf00715f0 0x7fbaf00c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.000+0000 7fbaf737c700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbad406ace0 0x7fbad406d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.000+0000 7fbaf737c700 1 --2- 172.21.15.22:0/3921322017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaf00d0300 0x7fbaf0156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.000+0000 7fbaf737c700 1 -- 172.21.15.22:0/3921322017 >> 172.21.15.22:0/3921322017 conn(0x7fbaf006b110 msgr2=0x7fbaf006b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:04.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.001+0000 7fbaf737c700 1 -- 172.21.15.22:0/3921322017 shutdown_connections 2023-07-06T01:20:04.006 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.001+0000 7fbaf737c700 1 -- 172.21.15.22:0/3921322017 wait complete. 2023-07-06T01:20:04.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T01:19:22.873255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T01:19:24.019088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:20:04.016 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:20:04.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.141+0000 7fb65907a700 1 Processor -- start 2023-07-06T01:20:04.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.141+0000 7fb65907a700 1 -- start start 2023-07-06T01:20:04.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.142+0000 7fb65907a700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fb654153850 0x7fb65406a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.142+0000 7fb65907a700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65406aea0 con 0x7fb65406b320 2023-07-06T01:20:04.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.142+0000 7fb65907a700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65406a8e0 con 0x7fb654153480 2023-07-06T01:20:04.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.142+0000 7fb65907a700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65406abc0 con 0x7fb654153850 2023-07-06T01:20:04.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.142+0000 7fb65359e700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb65406b320 0x7fb65406c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37398/0 (socket says 172.21.15.22:37398) 2023-07-06T01:20:04.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.142+0000 7fb65359e700 1 -- 172.21.15.22:0/900368567 learned_addr learned my addr 172.21.15.22:0/900368567 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:04.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.142+0000 7fb65259c700 1 --2- 172.21.15.22:0/900368567 >> v2:172.21.15.103:3300/0 conn(0x7fb654153850 0x7fb65406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.142+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167817037 0 0) 0x7fb65406aea0 con 0x7fb65406b320 2023-07-06T01:20:04.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.142+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb63c0035b0 con 0x7fb65406b320 2023-07-06T01:20:04.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 508580061 0 0) 0x7fb63c0035b0 con 0x7fb65406b320 2023-07-06T01:20:04.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 >> v2:172.21.15.103:3300/0 conn(0x7fb654153850 msgr2=0x7fb65406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:04.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 --2- 172.21.15.22:0/900368567 >> v2:172.21.15.103:3300/0 conn(0x7fb654153850 0x7fb65406a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 >> v1:172.21.15.103:6789/0 conn(0x7fb654153480 legacy=0x7fb65415d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65415cd80 con 0x7fb65406b320 2023-07-06T01:20:04.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7fb64c0019f0 con 0x7fb65406b320 2023-07-06T01:20:04.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 >> v1:172.21.15.22:6789/0 conn(0x7fb65406b320 legacy=0x7fb65406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 --2- 172.21.15.22:0/900368567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb63c003a00 0x7fb63c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 --2- 172.21.15.22:0/900368567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c0043f0 0x7fb63c0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 --2- 172.21.15.22:0/900368567 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb63c0050b0 0x7fb63c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65406abc0 con 0x7fb63c0043f0 2023-07-06T01:20:04.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c00c000 con 0x7fb63c003a00 2023-07-06T01:20:04.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c00c1c0 con 0x7fb63c0050b0 2023-07-06T01:20:04.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb65259c700 1 --2- 172.21.15.22:0/900368567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c0043f0 0x7fb63c0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb652d9d700 1 --2- 172.21.15.22:0/900368567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb63c003a00 0x7fb63c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb65359e700 1 --2- 172.21.15.22:0/900368567 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb63c0050b0 0x7fb63c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb65259c700 1 -- 172.21.15.22:0/900368567 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb63c0050b0 msgr2=0x7fb63c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:04.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb65259c700 1 --2- 172.21.15.22:0/900368567 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb63c0050b0 0x7fb63c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb65259c700 1 -- 172.21.15.22:0/900368567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb63c003a00 msgr2=0x7fb63c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:04.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb65259c700 1 --2- 172.21.15.22:0/900368567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb63c003a00 0x7fb63c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb65259c700 1 -- 172.21.15.22:0/900368567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c0035b0 con 0x7fb63c0043f0 2023-07-06T01:20:04.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb65359e700 1 -- 172.21.15.22:0/900368567 reap_dead start 2023-07-06T01:20:04.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb652d9d700 1 --2- 172.21.15.22:0/900368567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb63c003a00 0x7fb63c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:04.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb65259c700 1 --2- 172.21.15.22:0/900368567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c0043f0 0x7fb63c0048b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb65406d480 tx=0x7fb63c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dbec2e716f5e2c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:04.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.143+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64400e6c0 con 0x7fb63c0043f0 2023-07-06T01:20:04.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.144+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64400e820 con 0x7fb63c0043f0 2023-07-06T01:20:04.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.144+0000 7fb651d9b700 1 -- 172.21.15.22:0/900368567 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64400ebb0 con 0x7fb63c0043f0 2023-07-06T01:20:04.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.144+0000 7fb65907a700 1 -- 172.21.15.22:0/900368567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c0043f0 msgr2=0x7fb63c0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.144+0000 7fb65907a700 1 --2- 172.21.15.22:0/900368567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c0043f0 0x7fb63c0048b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb65406d480 tx=0x7fb63c005880 comp rx=0 tx=0).stop 2023-07-06T01:20:04.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.144+0000 7fb65907a700 1 -- 172.21.15.22:0/900368567 shutdown_connections 2023-07-06T01:20:04.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.144+0000 7fb65907a700 1 --2- 172.21.15.22:0/900368567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb63c0043f0 0x7fb63c0048b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.144+0000 7fb65907a700 1 -- 172.21.15.22:0/900368567 >> 172.21.15.22:0/900368567 conn(0x7fb6540ccb00 msgr2=0x7fb6540cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:04.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.144+0000 7fb65907a700 1 -- 172.21.15.22:0/900368567 shutdown_connections 2023-07-06T01:20:04.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.144+0000 7fb65907a700 1 -- 172.21.15.22:0/900368567 wait complete. 2023-07-06T01:20:04.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.146+0000 7fb65907a700 1 Processor -- start 2023-07-06T01:20:04.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.146+0000 7fb65907a700 1 -- start start 2023-07-06T01:20:04.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.146+0000 7fb65907a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb63c0043f0 0x7fb6540b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.146+0000 7fb65907a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65415c480 0x7fb654152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.146+0000 7fb65907a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb65415f990 0x7fb654071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.146+0000 7fb65907a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6540b3910 con 0x7fb65415c480 2023-07-06T01:20:04.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.146+0000 7fb65907a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65415c9b0 con 0x7fb63c0043f0 2023-07-06T01:20:04.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.146+0000 7fb65907a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65415ebd0 con 0x7fb65415f990 2023-07-06T01:20:04.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.146+0000 7fb65259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65415c480 0x7fb654152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb65259c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65415c480 0x7fb654152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46664/0 (socket says 172.21.15.22:46664) 2023-07-06T01:20:04.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb65259c700 1 -- 172.21.15.22:0/3020886807 learned_addr learned my addr 172.21.15.22:0/3020886807 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb65359e700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb65415f990 0x7fb654071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb652d9d700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb63c0043f0 0x7fb6540b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb65259c700 1 -- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb65415f990 msgr2=0x7fb654071210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb65259c700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb65415f990 0x7fb654071210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb65259c700 1 -- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb63c0043f0 msgr2=0x7fb6540b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb65259c700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb63c0043f0 0x7fb6540b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb65259c700 1 -- 172.21.15.22:0/3020886807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65415eeb0 con 0x7fb65415c480 2023-07-06T01:20:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb65359e700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb65415f990 0x7fb654071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb652d9d700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb63c0043f0 0x7fb6540b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb65259c700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65415c480 0x7fb654152190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb63c005810 tx=0x7fb6440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb63b7fe700 1 -- 172.21.15.22:0/3020886807 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644006e60 con 0x7fb65415c480 2023-07-06T01:20:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb63b7fe700 1 -- 172.21.15.22:0/3020886807 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb644013e30 con 0x7fb65415c480 2023-07-06T01:20:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb65907a700 1 -- 172.21.15.22:0/3020886807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65415f190 con 0x7fb65415c480 2023-07-06T01:20:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb63b7fe700 1 -- 172.21.15.22:0/3020886807 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64401c690 con 0x7fb65415c480 2023-07-06T01:20:04.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.147+0000 7fb65907a700 1 -- 172.21.15.22:0/3020886807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6541526d0 con 0x7fb65415c480 2023-07-06T01:20:04.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.148+0000 7fb65907a700 1 -- 172.21.15.22:0/3020886807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65415f7e0 con 0x7fb65415c480 2023-07-06T01:20:04.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.149+0000 7fb63b7fe700 1 -- 172.21.15.22:0/3020886807 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb64401c7f0 con 0x7fb65415c480 2023-07-06T01:20:04.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.149+0000 7fb63b7fe700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb64006ace0 0x7fb64006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.149+0000 7fb63b7fe700 1 -- 172.21.15.22:0/3020886807 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb64408ad40 con 0x7fb65415c480 2023-07-06T01:20:04.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.149+0000 7fb652d9d700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb64006ace0 0x7fb64006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.150+0000 7fb652d9d700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb64006ace0 0x7fb64006d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb654063c30 tx=0x7fb648009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:04.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.153+0000 7fb63b7fe700 1 -- 172.21.15.22:0/3020886807 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb644055fa0 con 0x7fb65415c480 2023-07-06T01:20:04.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.353+0000 7fb65907a700 1 -- 172.21.15.22:0/3020886807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb6540744b0 con 0x7fb65415c480 2023-07-06T01:20:04.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.353+0000 7fb63b7fe700 1 -- 172.21.15.22:0/3020886807 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb644017020 con 0x7fb65415c480 2023-07-06T01:20:04.356 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:20:04.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.355+0000 7fb65907a700 1 -- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb64006ace0 msgr2=0x7fb64006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.355+0000 7fb65907a700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb64006ace0 0x7fb64006d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb654063c30 tx=0x7fb648009040 comp rx=0 tx=0).stop 2023-07-06T01:20:04.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.355+0000 7fb65907a700 1 -- 172.21.15.22:0/3020886807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65415c480 msgr2=0x7fb654152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.355+0000 7fb65907a700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65415c480 0x7fb654152190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb63c005810 tx=0x7fb6440047c0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.355+0000 7fb65907a700 1 -- 172.21.15.22:0/3020886807 shutdown_connections 2023-07-06T01:20:04.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.355+0000 7fb65907a700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb65415f990 0x7fb654071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.355+0000 7fb65907a700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb64006ace0 0x7fb64006d190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.355+0000 7fb65907a700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb63c0043f0 0x7fb6540b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.355+0000 7fb65907a700 1 --2- 172.21.15.22:0/3020886807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb65415c480 0x7fb654152190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.355+0000 7fb65907a700 1 -- 172.21.15.22:0/3020886807 >> 172.21.15.22:0/3020886807 conn(0x7fb6540ccb00 msgr2=0x7fb65406c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.355+0000 7fb65907a700 1 -- 172.21.15.22:0/3020886807 shutdown_connections 2023-07-06T01:20:04.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.355+0000 7fb65907a700 1 -- 172.21.15.22:0/3020886807 wait complete. 2023-07-06T01:20:04.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:20:04.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.496+0000 7f77d65a8700 1 Processor -- start 2023-07-06T01:20:04.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.496+0000 7f77d65a8700 1 -- start start 2023-07-06T01:20:04.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77d65a8700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f77d00af2a0 0x7f77d006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77d65a8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f77d00cc320 0x7f77d006a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77d65a8700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d00af670 con 0x7f77d006a000 2023-07-06T01:20:04.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77d65a8700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d006a9e0 con 0x7f77d00cc320 2023-07-06T01:20:04.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77d65a8700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d00cd560 con 0x7f77d00af2a0 2023-07-06T01:20:04.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77d4b45700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f77d00cc320 0x7f77d006a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77cf7fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f77d006a000 0x7f77d00cbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37414/0 (socket says 172.21.15.22:37414) 2023-07-06T01:20:04.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77cf7fe700 1 -- 172.21.15.22:0/3760181419 learned_addr learned my addr 172.21.15.22:0/3760181419 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:04.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77cffff700 1 --2- 172.21.15.22:0/3760181419 >> v2:172.21.15.103:3300/0 conn(0x7f77d00af2a0 0x7f77d006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77d4b45700 1 -- 172.21.15.22:0/3760181419 >> v2:172.21.15.103:3300/0 conn(0x7f77d00af2a0 msgr2=0x7f77d006caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:04.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77d4b45700 1 --2- 172.21.15.22:0/3760181419 >> v2:172.21.15.103:3300/0 conn(0x7f77d00af2a0 0x7f77d006caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77d4b45700 1 -- 172.21.15.22:0/3760181419 >> v1:172.21.15.22:6789/0 conn(0x7f77d006a000 legacy=0x7f77d00cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77d4b45700 1 -- 172.21.15.22:0/3760181419 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d006d030 con 0x7f77d00cc320 2023-07-06T01:20:04.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77d4b45700 1 --2- 172.21.15.22:0/3760181419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d00cc320 0x7f77d006a4a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f77d0067460 tx=0x7f77c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4ca450043d7500 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:04.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.497+0000 7f77ceffd700 1 -- 172.21.15.22:0/3760181419 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c800e070 con 0x7f77d00cc320 2023-07-06T01:20:04.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.498+0000 7f77ceffd700 1 -- 172.21.15.22:0/3760181419 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c800bbd0 con 0x7f77d00cc320 2023-07-06T01:20:04.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.498+0000 7f77ceffd700 1 -- 172.21.15.22:0/3760181419 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8012430 con 0x7f77d00cc320 2023-07-06T01:20:04.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.498+0000 7f77d65a8700 1 -- 172.21.15.22:0/3760181419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d00cc320 msgr2=0x7f77d006a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.498+0000 7f77d65a8700 1 --2- 172.21.15.22:0/3760181419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d00cc320 0x7f77d006a4a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f77d0067460 tx=0x7f77c800b330 comp rx=0 tx=0).stop 2023-07-06T01:20:04.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.498+0000 7f77d65a8700 1 -- 172.21.15.22:0/3760181419 shutdown_connections 2023-07-06T01:20:04.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.498+0000 7f77d65a8700 1 --2- 172.21.15.22:0/3760181419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d00cc320 0x7f77d006a4a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.498+0000 7f77d65a8700 1 --2- 172.21.15.22:0/3760181419 >> v2:172.21.15.103:3300/0 conn(0x7f77d00af2a0 0x7f77d006caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.498+0000 7f77d65a8700 1 -- 172.21.15.22:0/3760181419 >> 172.21.15.22:0/3760181419 conn(0x7f77d00b1890 msgr2=0x7f77d00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:04.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.498+0000 7f77d65a8700 1 -- 172.21.15.22:0/3760181419 shutdown_connections 2023-07-06T01:20:04.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.498+0000 7f77d65a8700 1 -- 172.21.15.22:0/3760181419 wait complete. 2023-07-06T01:20:04.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.499+0000 7f77d65a8700 1 Processor -- start 2023-07-06T01:20:04.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.500+0000 7f77d65a8700 1 -- start start 2023-07-06T01:20:04.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.500+0000 7f77d65a8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77d006a000 0x7f77d01469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.500+0000 7f77d65a8700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f77d00af2a0 0x7f77d00c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.500+0000 7f77d65a8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d00cc320 0x7f77d00c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.500+0000 7f77d65a8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00c96a0 con 0x7f77d00cc320 2023-07-06T01:20:04.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.500+0000 7f77d65a8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01f1f70 con 0x7f77d006a000 2023-07-06T01:20:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.500+0000 7f77d65a8700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d0149fb0 con 0x7f77d00af2a0 2023-07-06T01:20:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.500+0000 7f77d4b45700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d00cc320 0x7f77d00c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.500+0000 7f77d4b45700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d00cc320 0x7f77d00c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46680/0 (socket says 172.21.15.22:46680) 2023-07-06T01:20:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.500+0000 7f77d4b45700 1 -- 172.21.15.22:0/700154796 learned_addr learned my addr 172.21.15.22:0/700154796 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:04.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.501+0000 7f77cf7fe700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f77d00af2a0 0x7f77d00c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.501+0000 7f77cffff700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77d006a000 0x7f77d01469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.501+0000 7f77d4b45700 1 -- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f77d00af2a0 msgr2=0x7f77d00c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.501+0000 7f77d4b45700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f77d00af2a0 0x7f77d00c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.501+0000 7f77d4b45700 1 -- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77d006a000 msgr2=0x7f77d01469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.501+0000 7f77d4b45700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77d006a000 0x7f77d01469a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.501+0000 7f77d4b45700 1 -- 172.21.15.22:0/700154796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c8009a70 con 0x7f77d00cc320 2023-07-06T01:20:04.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.501+0000 7f77d4b45700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d00cc320 0x7f77d00c8810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f77d00cd820 tx=0x7f77c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.501+0000 7f77ccff9700 1 -- 172.21.15.22:0/700154796 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8012430 con 0x7f77d00cc320 2023-07-06T01:20:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.501+0000 7f77ccff9700 1 -- 172.21.15.22:0/700154796 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c8020070 con 0x7f77d00cc320 2023-07-06T01:20:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.501+0000 7f77d65a8700 1 -- 172.21.15.22:0/700154796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d014a290 con 0x7f77d00cc320 2023-07-06T01:20:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.501+0000 7f77ccff9700 1 -- 172.21.15.22:0/700154796 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8004280 con 0x7f77d00cc320 2023-07-06T01:20:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.501+0000 7f77d65a8700 1 -- 172.21.15.22:0/700154796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d0147690 con 0x7f77d00cc320 2023-07-06T01:20:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.502+0000 7f77d65a8700 1 -- 172.21.15.22:0/700154796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d015ca90 con 0x7f77d00cc320 2023-07-06T01:20:04.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.502+0000 7f77ccff9700 1 -- 172.21.15.22:0/700154796 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f77c80043e0 con 0x7f77d00cc320 2023-07-06T01:20:04.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.503+0000 7f77ccff9700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f77b006ace0 0x7f77b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.503+0000 7f77ccff9700 1 -- 172.21.15.22:0/700154796 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f77c8022070 con 0x7f77d00cc320 2023-07-06T01:20:04.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.503+0000 7f77cffff700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f77b006ace0 0x7f77b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.504+0000 7f77cffff700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f77b006ace0 0x7f77b006d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f77d015fd50 tx=0x7f77c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:04.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.507+0000 7f77ccff9700 1 -- 172.21.15.22:0/700154796 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f77d015ca90 con 0x7f77d00cc320 2023-07-06T01:20:04.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.705+0000 7f77d65a8700 1 -- 172.21.15.22:0/700154796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77d015ca90 con 0x7f77d00cc320 2023-07-06T01:20:04.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.706+0000 7f77ccff9700 1 -- 172.21.15.22:0/700154796 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77d015ca90 con 0x7f77d00cc320 2023-07-06T01:20:04.709 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:20:04.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.708+0000 7f77d65a8700 1 -- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f77b006ace0 msgr2=0x7f77b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.708+0000 7f77d65a8700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f77b006ace0 0x7f77b006d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f77d015fd50 tx=0x7f77c0008040 comp rx=0 tx=0).stop 2023-07-06T01:20:04.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.708+0000 7f77d65a8700 1 -- 172.21.15.22:0/700154796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d00cc320 msgr2=0x7f77d00c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.708+0000 7f77d65a8700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d00cc320 0x7f77d00c8810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f77d00cd820 tx=0x7f77c8004ad0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.708+0000 7f77d65a8700 1 -- 172.21.15.22:0/700154796 shutdown_connections 2023-07-06T01:20:04.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.708+0000 7f77d65a8700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77d00cc320 0x7f77d00c8810 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.708+0000 7f77d65a8700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f77b006ace0 0x7f77b006d190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.708+0000 7f77d65a8700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77d006a000 0x7f77d01469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.708+0000 7f77d65a8700 1 --2- 172.21.15.22:0/700154796 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f77d00af2a0 0x7f77d00c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.708+0000 7f77d65a8700 1 -- 172.21.15.22:0/700154796 >> 172.21.15.22:0/700154796 conn(0x7f77d00b1890 msgr2=0x7f77d00b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:04.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.708+0000 7f77d65a8700 1 -- 172.21.15.22:0/700154796 shutdown_connections 2023-07-06T01:20:04.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.708+0000 7f77d65a8700 1 -- 172.21.15.22:0/700154796 wait complete. 2023-07-06T01:20:04.722 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T01:20:04.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.848+0000 7f104cae5700 1 Processor -- start 2023-07-06T01:20:04.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.848+0000 7f104cae5700 1 -- start start 2023-07-06T01:20:04.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.849+0000 7f104cae5700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f104815a710 0x7f10480ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.849+0000 7f104cae5700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f10480aecb0 0x7f10480ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.849+0000 7f104cae5700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10480af270 con 0x7f10480ceda0 2023-07-06T01:20:04.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.849+0000 7f104cae5700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10480cf220 con 0x7f10480aecb0 2023-07-06T01:20:04.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.849+0000 7f104cae5700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10480af550 con 0x7f104815a710 2023-07-06T01:20:04.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.849+0000 7f1045d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f10480aecb0 0x7f10480ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.849+0000 7f1045d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f10480aecb0 0x7f10480ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46684/0 (socket says 172.21.15.22:46684) 2023-07-06T01:20:04.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.849+0000 7f1045d9b700 1 -- 172.21.15.22:0/2594303924 learned_addr learned my addr 172.21.15.22:0/2594303924 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.849+0000 7f104659c700 1 --2- 172.21.15.22:0/2594303924 >> v2:172.21.15.103:3300/0 conn(0x7f104815a710 0x7f10480ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.849+0000 7f1045d9b700 1 -- 172.21.15.22:0/2594303924 >> v2:172.21.15.103:3300/0 conn(0x7f104815a710 msgr2=0x7f10480ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.849+0000 7f1045d9b700 1 --2- 172.21.15.22:0/2594303924 >> v2:172.21.15.103:3300/0 conn(0x7f104815a710 0x7f10480ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.849+0000 7f1045d9b700 1 -- 172.21.15.22:0/2594303924 >> v1:172.21.15.103:6789/0 conn(0x7f10480ceda0 legacy=0x7f104815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.849+0000 7f1045d9b700 1 -- 172.21.15.22:0/2594303924 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10480af830 con 0x7f10480aecb0 2023-07-06T01:20:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.850+0000 7f1045d9b700 1 --2- 172.21.15.22:0/2594303924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10480aecb0 0x7f10480ce860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1048070f00 tx=0x7f1038009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cbd90dd4fff07d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.850+0000 7f104559a700 1 -- 172.21.15.22:0/2594303924 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103800b870 con 0x7f10480aecb0 2023-07-06T01:20:04.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.850+0000 7f104559a700 1 -- 172.21.15.22:0/2594303924 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103800b9d0 con 0x7f10480aecb0 2023-07-06T01:20:04.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.850+0000 7f104559a700 1 -- 172.21.15.22:0/2594303924 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103800bd60 con 0x7f10480aecb0 2023-07-06T01:20:04.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.850+0000 7f104cae5700 1 -- 172.21.15.22:0/2594303924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10480aecb0 msgr2=0x7f10480ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.850+0000 7f104cae5700 1 --2- 172.21.15.22:0/2594303924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10480aecb0 0x7f10480ce860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1048070f00 tx=0x7f1038009bd0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.850+0000 7f104cae5700 1 -- 172.21.15.22:0/2594303924 shutdown_connections 2023-07-06T01:20:04.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.850+0000 7f104cae5700 1 --2- 172.21.15.22:0/2594303924 >> v2:172.21.15.103:3300/0 conn(0x7f104815a710 0x7f10480ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.850+0000 7f104cae5700 1 --2- 172.21.15.22:0/2594303924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10480aecb0 0x7f10480ce860 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.850+0000 7f104cae5700 1 -- 172.21.15.22:0/2594303924 >> 172.21.15.22:0/2594303924 conn(0x7f10480999f0 msgr2=0x7f1048099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:04.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.851+0000 7f104cae5700 1 -- 172.21.15.22:0/2594303924 shutdown_connections 2023-07-06T01:20:04.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.851+0000 7f104cae5700 1 -- 172.21.15.22:0/2594303924 wait complete. 2023-07-06T01:20:04.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.852+0000 7f104cae5700 1 Processor -- start 2023-07-06T01:20:04.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.852+0000 7f104cae5700 1 -- start start 2023-07-06T01:20:04.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104cae5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10480aecb0 0x7f1048156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104cae5700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f10480ceda0 0x7f10481542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104cae5700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f104815a710 0x7f10480c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104cae5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1048154830 con 0x7f10480aecb0 2023-07-06T01:20:04.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104cae5700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10481f35b0 con 0x7f104815a710 2023-07-06T01:20:04.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104cae5700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10481f3890 con 0x7f10480ceda0 2023-07-06T01:20:04.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10480aecb0 0x7f1048156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10480aecb0 0x7f1048156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46688/0 (socket says 172.21.15.22:46688) 2023-07-06T01:20:04.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104659c700 1 -- 172.21.15.22:0/2312027581 learned_addr learned my addr 172.21.15.22:0/2312027581 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:04.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f1046d9d700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f104815a710 0x7f10480c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f1045d9b700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f10480ceda0 0x7f10481542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104659c700 1 -- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f10480ceda0 msgr2=0x7f10481542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104659c700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f10480ceda0 0x7f10481542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104659c700 1 -- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f104815a710 msgr2=0x7f10480c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:04.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104659c700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f104815a710 0x7f10480c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:04.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104659c700 1 -- 172.21.15.22:0/2312027581 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1038009550 con 0x7f10480aecb0 2023-07-06T01:20:04.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f1045d9b700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f10480ceda0 0x7f10481542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:04.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104659c700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10480aecb0 0x7f1048156350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f103c00e810 tx=0x7f103c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:04.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f1036ffd700 1 -- 172.21.15.22:0/2312027581 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c013610 con 0x7f10480aecb0 2023-07-06T01:20:04.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f1036ffd700 1 -- 172.21.15.22:0/2312027581 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c0137b0 con 0x7f10480aecb0 2023-07-06T01:20:04.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f104cae5700 1 -- 172.21.15.22:0/2312027581 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10481f3b70 con 0x7f10480aecb0 2023-07-06T01:20:04.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.853+0000 7f1036ffd700 1 -- 172.21.15.22:0/2312027581 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c013b60 con 0x7f10480aecb0 2023-07-06T01:20:04.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.854+0000 7f104cae5700 1 -- 172.21.15.22:0/2312027581 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10480c9440 con 0x7f10480aecb0 2023-07-06T01:20:04.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.854+0000 7f104cae5700 1 -- 172.21.15.22:0/2312027581 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10480cb2c0 con 0x7f10480aecb0 2023-07-06T01:20:04.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.855+0000 7f1036ffd700 1 -- 172.21.15.22:0/2312027581 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f103c013cc0 con 0x7f10480aecb0 2023-07-06T01:20:04.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.855+0000 7f1036ffd700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f102806ace0 0x7f102806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:04.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.855+0000 7f1036ffd700 1 -- 172.21.15.22:0/2312027581 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f103c089160 con 0x7f10480aecb0 2023-07-06T01:20:04.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.856+0000 7f1045d9b700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f102806ace0 0x7f102806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:04.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.856+0000 7f1045d9b700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f102806ace0 0x7f102806d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1038000c00 tx=0x7f1038008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:04.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:04.860+0000 7f1036ffd700 1 -- 172.21.15.22:0/2312027581 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f10480cb2c0 con 0x7f10480aecb0 2023-07-06T01:20:05.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.057+0000 7f104cae5700 1 -- 172.21.15.22:0/2312027581 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10480cb2c0 con 0x7f10480aecb0 2023-07-06T01:20:05.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.057+0000 7f1036ffd700 1 -- 172.21.15.22:0/2312027581 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f10480cb2c0 con 0x7f10480aecb0 2023-07-06T01:20:05.060 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-07-06T01:20:05.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.059+0000 7f104cae5700 1 -- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f102806ace0 msgr2=0x7f102806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.059+0000 7f104cae5700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f102806ace0 0x7f102806d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1038000c00 tx=0x7f1038008030 comp rx=0 tx=0).stop 2023-07-06T01:20:05.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.059+0000 7f104cae5700 1 -- 172.21.15.22:0/2312027581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10480aecb0 msgr2=0x7f1048156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.059+0000 7f104cae5700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10480aecb0 0x7f1048156350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f103c00e810 tx=0x7f103c00eb20 comp rx=0 tx=0).stop 2023-07-06T01:20:05.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.059+0000 7f104cae5700 1 -- 172.21.15.22:0/2312027581 shutdown_connections 2023-07-06T01:20:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.059+0000 7f104cae5700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f102806ace0 0x7f102806d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.059+0000 7f104cae5700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f104815a710 0x7f10480c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.059+0000 7f104cae5700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10480aecb0 0x7f1048156350 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.059+0000 7f104cae5700 1 --2- 172.21.15.22:0/2312027581 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f10480ceda0 0x7f10481542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.059+0000 7f104cae5700 1 -- 172.21.15.22:0/2312027581 >> 172.21.15.22:0/2312027581 conn(0x7f10480999f0 msgr2=0x7f10480c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.059+0000 7f104cae5700 1 -- 172.21.15.22:0/2312027581 shutdown_connections 2023-07-06T01:20:05.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.059+0000 7f104cae5700 1 -- 172.21.15.22:0/2312027581 wait complete. 2023-07-06T01:20:05.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T01:20:05.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.199+0000 7f9b0954a700 1 Processor -- start 2023-07-06T01:20:05.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.199+0000 7f9b0954a700 1 -- start start 2023-07-06T01:20:05.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.200+0000 7f9b0954a700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9b040b3b70 0x7f9b040d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:05.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.200+0000 7f9b0954a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b040b40c0 0x7f9b040d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:05.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.200+0000 7f9b0954a700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b040cebd0 con 0x7f9b040b36f0 2023-07-06T01:20:05.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.200+0000 7f9b0954a700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0415cff0 con 0x7f9b040b3b70 2023-07-06T01:20:05.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.200+0000 7f9b0954a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040cdfc0 con 0x7f9b040b40c0 2023-07-06T01:20:05.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.200+0000 7f9b02ffd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9b040b36f0 0x7f9b040cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37440/0 (socket says 172.21.15.22:37440) 2023-07-06T01:20:05.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.200+0000 7f9b02ffd700 1 -- 172.21.15.22:0/2142706877 learned_addr learned my addr 172.21.15.22:0/2142706877 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:05.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.200+0000 7f9b037fe700 1 --2- 172.21.15.22:0/2142706877 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b040b40c0 0x7f9b040d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:05.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.200+0000 7f9b027fc700 1 --2- 172.21.15.22:0/2142706877 >> v2:172.21.15.103:3300/0 conn(0x7f9b040b3b70 0x7f9b040d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:05.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.200+0000 7f9b01ffb700 1 -- 172.21.15.22:0/2142706877 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693217641 0 0) 0x7f9b040cebd0 con 0x7f9b040b36f0 2023-07-06T01:20:05.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.200+0000 7f9b01ffb700 1 -- 172.21.15.22:0/2142706877 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aec0035b0 con 0x7f9b040b36f0 2023-07-06T01:20:05.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.200+0000 7f9b01ffb700 1 -- 172.21.15.22:0/2142706877 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2630477534 0 0) 0x7f9aec0035b0 con 0x7f9b040b36f0 2023-07-06T01:20:05.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.200+0000 7f9b027fc700 1 -- 172.21.15.22:0/2142706877 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b040b40c0 msgr2=0x7f9b040d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:05.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.201+0000 7f9b027fc700 1 --2- 172.21.15.22:0/2142706877 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b040b40c0 0x7f9b040d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.201+0000 7f9b027fc700 1 -- 172.21.15.22:0/2142706877 >> v1:172.21.15.22:6789/0 conn(0x7f9b040b36f0 legacy=0x7f9b040cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.201+0000 7f9b027fc700 1 -- 172.21.15.22:0/2142706877 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b040ce2a0 con 0x7f9b040b3b70 2023-07-06T01:20:05.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.201+0000 7f9b037fe700 1 --2- 172.21.15.22:0/2142706877 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b040b40c0 0x7f9b040d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:05.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.201+0000 7f9b027fc700 1 --2- 172.21.15.22:0/2142706877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b040b3b70 0x7f9b040d0a90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009840 tx=0x7f9af4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e100605d3b6f2ad0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:05.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.201+0000 7f9b01ffb700 1 -- 172.21.15.22:0/2142706877 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af8001670 con 0x7f9b040b3b70 2023-07-06T01:20:05.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.201+0000 7f9b01ffb700 1 -- 172.21.15.22:0/2142706877 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af4004030 con 0x7f9b040b3b70 2023-07-06T01:20:05.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.201+0000 7f9b01ffb700 1 -- 172.21.15.22:0/2142706877 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af40043c0 con 0x7f9b040b3b70 2023-07-06T01:20:05.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.201+0000 7f9b0954a700 1 -- 172.21.15.22:0/2142706877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b040b3b70 msgr2=0x7f9b040d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.201+0000 7f9b0954a700 1 --2- 172.21.15.22:0/2142706877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b040b3b70 0x7f9b040d0a90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009840 tx=0x7f9af4009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:05.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.201+0000 7f9b0954a700 1 -- 172.21.15.22:0/2142706877 shutdown_connections 2023-07-06T01:20:05.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.201+0000 7f9b0954a700 1 --2- 172.21.15.22:0/2142706877 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b040b40c0 0x7f9b040d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.202+0000 7f9b0954a700 1 --2- 172.21.15.22:0/2142706877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b040b3b70 0x7f9b040d0a90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.202+0000 7f9b0954a700 1 -- 172.21.15.22:0/2142706877 >> 172.21.15.22:0/2142706877 conn(0x7f9b0409c2e0 msgr2=0x7f9b040d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:05.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.202+0000 7f9b0954a700 1 -- 172.21.15.22:0/2142706877 shutdown_connections 2023-07-06T01:20:05.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.202+0000 7f9b0954a700 1 -- 172.21.15.22:0/2142706877 wait complete. 2023-07-06T01:20:05.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.203+0000 7f9b0954a700 1 Processor -- start 2023-07-06T01:20:05.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.203+0000 7f9b0954a700 1 -- start start 2023-07-06T01:20:05.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b0954a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b040b36f0 0x7f9b040c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:05.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b0954a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b040b3b70 0x7f9b040cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:05.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b0954a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b040b40c0 0x7f9b040c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:05.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b0954a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b04154c50 con 0x7f9b040b3b70 2023-07-06T01:20:05.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b0954a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040c7790 con 0x7f9b040b40c0 2023-07-06T01:20:05.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b0954a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b04154ef0 con 0x7f9b040b36f0 2023-07-06T01:20:05.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b027fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b040b3b70 0x7f9b040cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:05.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b027fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b040b3b70 0x7f9b040cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46704/0 (socket says 172.21.15.22:46704) 2023-07-06T01:20:05.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b027fc700 1 -- 172.21.15.22:0/970314956 learned_addr learned my addr 172.21.15.22:0/970314956 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:05.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b037fe700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b040b40c0 0x7f9b040c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:05.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b02ffd700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b040b36f0 0x7f9b040c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:05.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b027fc700 1 -- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b040b36f0 msgr2=0x7f9b040c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b027fc700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b040b36f0 0x7f9b040c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b027fc700 1 -- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b040b40c0 msgr2=0x7f9b040c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b027fc700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b040b40c0 0x7f9b040c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b027fc700 1 -- 172.21.15.22:0/970314956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc009a70 con 0x7f9b040b3b70 2023-07-06T01:20:05.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.204+0000 7f9b027fc700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b040b3b70 0x7f9b040cbdc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9af400a010 tx=0x7f9af4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:05.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.205+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/970314956 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4003a90 con 0x7f9b040b3b70 2023-07-06T01:20:05.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.205+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/970314956 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af401f070 con 0x7f9b040b3b70 2023-07-06T01:20:05.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.205+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/970314956 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4003e50 con 0x7f9b040b3b70 2023-07-06T01:20:05.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.205+0000 7f9b0954a700 1 -- 172.21.15.22:0/970314956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af40094a0 con 0x7f9b040b3b70 2023-07-06T01:20:05.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.205+0000 7f9b0954a700 1 -- 172.21.15.22:0/970314956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b041554b0 con 0x7f9b040b3b70 2023-07-06T01:20:05.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.205+0000 7f9b0954a700 1 -- 172.21.15.22:0/970314956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b040b4840 con 0x7f9b040b3b70 2023-07-06T01:20:05.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.206+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/970314956 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9af4021030 con 0x7f9b040b3b70 2023-07-06T01:20:05.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.206+0000 7f9aeb7fe700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9af006ace0 0x7f9af006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:05.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.206+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/970314956 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9af408a6b0 con 0x7f9b040b3b70 2023-07-06T01:20:05.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.207+0000 7f9b02ffd700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9af006ace0 0x7f9af006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:05.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.207+0000 7f9b02ffd700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9af006ace0 0x7f9af006d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9af8000f60 tx=0x7f9af8001e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:05.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.211+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/970314956 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9af4055910 con 0x7f9b040b3b70 2023-07-06T01:20:05.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.412+0000 7f9b0954a700 1 -- 172.21.15.22:0/970314956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9b040744b0 con 0x7f9b040b3b70 2023-07-06T01:20:05.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.413+0000 7f9aeb7fe700 1 -- 172.21.15.22:0/970314956 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9af4058f30 con 0x7f9b040b3b70 2023-07-06T01:20:05.416 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-07-06T01:20:05.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.415+0000 7f9b0954a700 1 -- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9af006ace0 msgr2=0x7f9af006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.415+0000 7f9b0954a700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9af006ace0 0x7f9af006d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9af8000f60 tx=0x7f9af8001e00 comp rx=0 tx=0).stop 2023-07-06T01:20:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.415+0000 7f9b0954a700 1 -- 172.21.15.22:0/970314956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b040b3b70 msgr2=0x7f9b040cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.415+0000 7f9b0954a700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b040b3b70 0x7f9b040cbdc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9af400a010 tx=0x7f9af4004750 comp rx=0 tx=0).stop 2023-07-06T01:20:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.415+0000 7f9b0954a700 1 -- 172.21.15.22:0/970314956 shutdown_connections 2023-07-06T01:20:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.415+0000 7f9b0954a700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9af006ace0 0x7f9af006d190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.415+0000 7f9b0954a700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b040b40c0 0x7f9b040c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.415+0000 7f9b0954a700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b040b36f0 0x7f9b040c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.420 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.415+0000 7f9b0954a700 1 --2- 172.21.15.22:0/970314956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b040b3b70 0x7f9b040cbdc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.415+0000 7f9b0954a700 1 -- 172.21.15.22:0/970314956 >> 172.21.15.22:0/970314956 conn(0x7f9b0409c2e0 msgr2=0x7f9b040b8260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:05.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.415+0000 7f9b0954a700 1 -- 172.21.15.22:0/970314956 shutdown_connections 2023-07-06T01:20:05.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.415+0000 7f9b0954a700 1 -- 172.21.15.22:0/970314956 wait complete. 2023-07-06T01:20:05.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T01:20:05.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.553+0000 7fe54c5ca700 1 Processor -- start 2023-07-06T01:20:05.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.553+0000 7fe54c5ca700 1 -- start start 2023-07-06T01:20:05.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.554+0000 7fe54c5ca700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe544159360 0x7fe54415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:05.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.554+0000 7fe54c5ca700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe5440cc660 0x7fe5440ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:05.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.554+0000 7fe54c5ca700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5440cf230 con 0x7fe544070a20 2023-07-06T01:20:05.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.554+0000 7fe54c5ca700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54415df20 con 0x7fe5440cc660 2023-07-06T01:20:05.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.554+0000 7fe54c5ca700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5440cd020 con 0x7fe544159360 2023-07-06T01:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.554+0000 7fe549b65700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe5440cc660 0x7fe5440ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.554+0000 7fe54ab67700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe544070a20 0x7fe544070e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37458/0 (socket says 172.21.15.22:37458) 2023-07-06T01:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.554+0000 7fe54ab67700 1 -- 172.21.15.22:0/932289337 learned_addr learned my addr 172.21.15.22:0/932289337 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.555+0000 7fe54a366700 1 --2- 172.21.15.22:0/932289337 >> v2:172.21.15.103:3300/0 conn(0x7fe544159360 0x7fe54415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.555+0000 7fe549b65700 1 -- 172.21.15.22:0/932289337 >> v2:172.21.15.103:3300/0 conn(0x7fe544159360 msgr2=0x7fe54415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.555+0000 7fe549b65700 1 --2- 172.21.15.22:0/932289337 >> v2:172.21.15.103:3300/0 conn(0x7fe544159360 0x7fe54415d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.555+0000 7fe549b65700 1 -- 172.21.15.22:0/932289337 >> v1:172.21.15.22:6789/0 conn(0x7fe544070a20 legacy=0x7fe544070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.555+0000 7fe549b65700 1 -- 172.21.15.22:0/932289337 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5440ce620 con 0x7fe5440cc660 2023-07-06T01:20:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.555+0000 7fe549364700 1 -- 172.21.15.22:0/932289337 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658068467 0 0) 0x7fe5440cf230 con 0x7fe544070a20 2023-07-06T01:20:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.555+0000 7fe54a366700 1 --2- 172.21.15.22:0/932289337 >> v2:172.21.15.103:3300/0 conn(0x7fe544159360 0x7fe54415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.555+0000 7fe549b65700 1 --2- 172.21.15.22:0/932289337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5440cc660 0x7fe5440ccae0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe544071aa0 tx=0x7fe540009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32119fcb7e0bf11 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.555+0000 7fe549364700 1 -- 172.21.15.22:0/932289337 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe540011830 con 0x7fe5440cc660 2023-07-06T01:20:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.555+0000 7fe549364700 1 -- 172.21.15.22:0/932289337 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe540011990 con 0x7fe5440cc660 2023-07-06T01:20:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.555+0000 7fe549364700 1 -- 172.21.15.22:0/932289337 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe540011d20 con 0x7fe5440cc660 2023-07-06T01:20:05.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.556+0000 7fe54c5ca700 1 -- 172.21.15.22:0/932289337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5440cc660 msgr2=0x7fe5440ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.556+0000 7fe54c5ca700 1 --2- 172.21.15.22:0/932289337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5440cc660 0x7fe5440ccae0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe544071aa0 tx=0x7fe540009b20 comp rx=0 tx=0).stop 2023-07-06T01:20:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.556+0000 7fe54c5ca700 1 -- 172.21.15.22:0/932289337 shutdown_connections 2023-07-06T01:20:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.556+0000 7fe54c5ca700 1 --2- 172.21.15.22:0/932289337 >> v2:172.21.15.103:3300/0 conn(0x7fe544159360 0x7fe54415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.556+0000 7fe54c5ca700 1 --2- 172.21.15.22:0/932289337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5440cc660 0x7fe5440ccae0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.556+0000 7fe54c5ca700 1 -- 172.21.15.22:0/932289337 >> 172.21.15.22:0/932289337 conn(0x7fe5440b1890 msgr2=0x7fe5440b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.556+0000 7fe54c5ca700 1 -- 172.21.15.22:0/932289337 shutdown_connections 2023-07-06T01:20:05.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.556+0000 7fe54c5ca700 1 -- 172.21.15.22:0/932289337 wait complete. 2023-07-06T01:20:05.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.557+0000 7fe54c5ca700 1 Processor -- start 2023-07-06T01:20:05.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.557+0000 7fe54c5ca700 1 -- start start 2023-07-06T01:20:05.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe54c5ca700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe544070a20 0x7fe5441487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:05.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe54c5ca700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5440cc660 0x7fe54415bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:05.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe54c5ca700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe544159360 0x7fe5440b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:05.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe54c5ca700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe544145400 con 0x7fe5440cc660 2023-07-06T01:20:05.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe54c5ca700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5441456e0 con 0x7fe544159360 2023-07-06T01:20:05.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe54c5ca700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5441459c0 con 0x7fe544070a20 2023-07-06T01:20:05.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe549b65700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5440cc660 0x7fe54415bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:05.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe549b65700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5440cc660 0x7fe54415bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46718/0 (socket says 172.21.15.22:46718) 2023-07-06T01:20:05.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe549b65700 1 -- 172.21.15.22:0/2233403880 learned_addr learned my addr 172.21.15.22:0/2233403880 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:05.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe54a366700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe544070a20 0x7fe5441487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:05.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe54ab67700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe544159360 0x7fe5440b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:05.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe549b65700 1 -- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe544070a20 msgr2=0x7fe5441487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe549b65700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe544070a20 0x7fe5441487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe549b65700 1 -- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe544159360 msgr2=0x7fe5440b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe549b65700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe544159360 0x7fe5440b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe549b65700 1 -- 172.21.15.22:0/2233403880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5400094a0 con 0x7fe5440cc660 2023-07-06T01:20:05.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe54a366700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe544070a20 0x7fe5441487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:05.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe549b65700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5440cc660 0x7fe54415bf60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe5400090b0 tx=0x7fe540004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:05.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe53affd700 1 -- 172.21.15.22:0/2233403880 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5400038a0 con 0x7fe5440cc660 2023-07-06T01:20:05.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe53affd700 1 -- 172.21.15.22:0/2233403880 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54001f070 con 0x7fe5440cc660 2023-07-06T01:20:05.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe53affd700 1 -- 172.21.15.22:0/2233403880 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe540003c20 con 0x7fe5440cc660 2023-07-06T01:20:05.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe54c5ca700 1 -- 172.21.15.22:0/2233403880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe544145ca0 con 0x7fe5440cc660 2023-07-06T01:20:05.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.558+0000 7fe54c5ca700 1 -- 172.21.15.22:0/2233403880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54415c4a0 con 0x7fe5440cc660 2023-07-06T01:20:05.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.559+0000 7fe54c5ca700 1 -- 172.21.15.22:0/2233403880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5441461c0 con 0x7fe5440cc660 2023-07-06T01:20:05.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.560+0000 7fe53affd700 1 -- 172.21.15.22:0/2233403880 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe54001a430 con 0x7fe5440cc660 2023-07-06T01:20:05.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.560+0000 7fe53affd700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe52806ace0 0x7fe52806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:05.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.560+0000 7fe53affd700 1 -- 172.21.15.22:0/2233403880 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe540021070 con 0x7fe5440cc660 2023-07-06T01:20:05.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.560+0000 7fe54a366700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe52806ace0 0x7fe52806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:05.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.561+0000 7fe54a366700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe52806ace0 0x7fe52806d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe5340099b0 tx=0x7fe534008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:05.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.565+0000 7fe53affd700 1 -- 172.21.15.22:0/2233403880 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe5441461c0 con 0x7fe5440cc660 2023-07-06T01:20:05.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.785+0000 7fe54c5ca700 1 -- 172.21.15.22:0/2233403880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe5441461c0 con 0x7fe5440cc660 2023-07-06T01:20:05.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.785+0000 7fe53affd700 1 -- 172.21.15.22:0/2233403880 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe5441461c0 con 0x7fe5440cc660 2023-07-06T01:20:05.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.787+0000 7fe54c5ca700 1 -- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe52806ace0 msgr2=0x7fe52806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.787+0000 7fe54c5ca700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe52806ace0 0x7fe52806d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe5340099b0 tx=0x7fe534008040 comp rx=0 tx=0).stop 2023-07-06T01:20:05.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.787+0000 7fe54c5ca700 1 -- 172.21.15.22:0/2233403880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5440cc660 msgr2=0x7fe54415bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:05.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.787+0000 7fe54c5ca700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5440cc660 0x7fe54415bf60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe5400090b0 tx=0x7fe540004750 comp rx=0 tx=0).stop 2023-07-06T01:20:05.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.787+0000 7fe54c5ca700 1 -- 172.21.15.22:0/2233403880 shutdown_connections 2023-07-06T01:20:05.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.787+0000 7fe54c5ca700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe52806ace0 0x7fe52806d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.787+0000 7fe54c5ca700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe544159360 0x7fe5440b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.787+0000 7fe54c5ca700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe544070a20 0x7fe5441487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.787+0000 7fe54c5ca700 1 --2- 172.21.15.22:0/2233403880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5440cc660 0x7fe54415bf60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:05.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.787+0000 7fe54c5ca700 1 -- 172.21.15.22:0/2233403880 >> 172.21.15.22:0/2233403880 conn(0x7fe5440b1890 msgr2=0x7fe5441581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:05.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.787+0000 7fe54c5ca700 1 -- 172.21.15.22:0/2233403880 shutdown_connections 2023-07-06T01:20:05.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:05.787+0000 7fe54c5ca700 1 -- 172.21.15.22:0/2233403880 wait complete. 2023-07-06T01:20:05.792 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-07-06T01:20:05.802 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-07-06T01:20:05.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:05.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-07-06T01:20:05.846 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:20:05.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:05.847 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T01:20:05.903 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:20:05.904 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:20:05.925 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:20:05.925 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:20:05.985 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:20:05.986 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:20:06.045 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:20:06.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.168+0000 7fd5f18b6700 1 Processor -- start 2023-07-06T01:20:06.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.168+0000 7fd5f18b6700 1 -- start start 2023-07-06T01:20:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.168+0000 7fd5f18b6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd5ec15c950 0x7fd5ec0cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.168+0000 7fd5f18b6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd5ec0d05e0 0x7fd5ec0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.168+0000 7fd5f18b6700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ec0cb650 con 0x7fd5ec06b110 2023-07-06T01:20:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.168+0000 7fd5f18b6700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ec06b5f0 con 0x7fd5ec15c950 2023-07-06T01:20:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.168+0000 7fd5f18b6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec06b8d0 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.169+0000 7fd5eaffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd5ec15c950 0x7fd5ec0cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.169+0000 7fd5eaffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fd5ec15c950 0x7fd5ec0cf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46724/0 (socket says 172.21.15.22:46724) 2023-07-06T01:20:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.169+0000 7fd5eaffd700 1 -- 172.21.15.22:0/2571298402 learned_addr learned my addr 172.21.15.22:0/2571298402 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.169+0000 7fd5eb7fe700 1 --2- 172.21.15.22:0/2571298402 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd5ec0d05e0 0x7fd5ec0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.169+0000 7fd5eaffd700 1 -- 172.21.15.22:0/2571298402 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd5ec0d05e0 msgr2=0x7fd5ec0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.169+0000 7fd5eaffd700 1 --2- 172.21.15.22:0/2571298402 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd5ec0d05e0 0x7fd5ec0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.169+0000 7fd5eaffd700 1 -- 172.21.15.22:0/2571298402 >> v1:172.21.15.103:6789/0 conn(0x7fd5ec06b110 legacy=0x7fd5ec0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:06.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.169+0000 7fd5eaffd700 1 -- 172.21.15.22:0/2571298402 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec06bbb0 con 0x7fd5ec15c950 2023-07-06T01:20:06.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.169+0000 7fd5eaffd700 1 --2- 172.21.15.22:0/2571298402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5ec15c950 0x7fd5ec0cf400 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc006fd0 tx=0x7fd5dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117a9fc7cd35db65 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:06.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.169+0000 7fd5e9ffb700 1 -- 172.21.15.22:0/2571298402 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc005e30 con 0x7fd5ec15c950 2023-07-06T01:20:06.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.169+0000 7fd5e9ffb700 1 -- 172.21.15.22:0/2571298402 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc004030 con 0x7fd5ec15c950 2023-07-06T01:20:06.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.169+0000 7fd5e9ffb700 1 -- 172.21.15.22:0/2571298402 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0043e0 con 0x7fd5ec15c950 2023-07-06T01:20:06.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.170+0000 7fd5f18b6700 1 -- 172.21.15.22:0/2571298402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5ec15c950 msgr2=0x7fd5ec0cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.170+0000 7fd5f18b6700 1 --2- 172.21.15.22:0/2571298402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5ec15c950 0x7fd5ec0cf400 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc006fd0 tx=0x7fd5dc009bd0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.170+0000 7fd5f18b6700 1 -- 172.21.15.22:0/2571298402 shutdown_connections 2023-07-06T01:20:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.170+0000 7fd5f18b6700 1 --2- 172.21.15.22:0/2571298402 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd5ec0d05e0 0x7fd5ec0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.170+0000 7fd5f18b6700 1 --2- 172.21.15.22:0/2571298402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5ec15c950 0x7fd5ec0cf400 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.170+0000 7fd5f18b6700 1 -- 172.21.15.22:0/2571298402 >> 172.21.15.22:0/2571298402 conn(0x7fd5ec0999f0 msgr2=0x7fd5ec099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.170+0000 7fd5f18b6700 1 -- 172.21.15.22:0/2571298402 shutdown_connections 2023-07-06T01:20:06.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.170+0000 7fd5f18b6700 1 -- 172.21.15.22:0/2571298402 wait complete. 2023-07-06T01:20:06.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.171+0000 7fd5f18b6700 1 Processor -- start 2023-07-06T01:20:06.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.171+0000 7fd5f18b6700 1 -- start start 2023-07-06T01:20:06.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.172+0000 7fd5f18b6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5ec06b110 0x7fd5ec155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.172+0000 7fd5f18b6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5ec0d05e0 0x7fd5ec0c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.172+0000 7fd5f18b6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd5ec15c950 0x7fd5ec0b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.172+0000 7fd5f18b6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0b4020 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.172+0000 7fd5f18b6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec1f26f0 con 0x7fd5ec06b110 2023-07-06T01:20:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.172+0000 7fd5f18b6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0c7620 con 0x7fd5ec15c950 2023-07-06T01:20:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.172+0000 7fd5ea7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5ec0d05e0 0x7fd5ec0c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.172+0000 7fd5ea7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5ec0d05e0 0x7fd5ec0c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46730/0 (socket says 172.21.15.22:46730) 2023-07-06T01:20:06.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.172+0000 7fd5ea7fc700 1 -- 172.21.15.22:0/410438949 learned_addr learned my addr 172.21.15.22:0/410438949 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:06.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.172+0000 7fd5eb7fe700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd5ec15c950 0x7fd5ec0b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.172+0000 7fd5eaffd700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5ec06b110 0x7fd5ec155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.173+0000 7fd5ea7fc700 1 -- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd5ec15c950 msgr2=0x7fd5ec0b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.173+0000 7fd5ea7fc700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd5ec15c950 0x7fd5ec0b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.173+0000 7fd5ea7fc700 1 -- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5ec06b110 msgr2=0x7fd5ec155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.173+0000 7fd5ea7fc700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5ec06b110 0x7fd5ec155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.173+0000 7fd5ea7fc700 1 -- 172.21.15.22:0/410438949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc009500 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.173+0000 7fd5ea7fc700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5ec0d05e0 0x7fd5ec0c60a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000c4e0 tx=0x7fd5e000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:06.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.173+0000 7fd5d3fff700 1 -- 172.21.15.22:0/410438949 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00052d0 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.173+0000 7fd5d3fff700 1 -- 172.21.15.22:0/410438949 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e0005430 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.173+0000 7fd5f18b6700 1 -- 172.21.15.22:0/410438949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ec0b3c80 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.173+0000 7fd5d3fff700 1 -- 172.21.15.22:0/410438949 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00038c0 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.173+0000 7fd5f18b6700 1 -- 172.21.15.22:0/410438949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec0c7340 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.174+0000 7fd5f18b6700 1 -- 172.21.15.22:0/410438949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ec0d0b20 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.174+0000 7fd5d3fff700 1 -- 172.21.15.22:0/410438949 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd5e0003a20 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.175+0000 7fd5d3fff700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd5d806ace0 0x7fd5d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.175+0000 7fd5d3fff700 1 -- 172.21.15.22:0/410438949 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd5e0088790 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.175+0000 7fd5eaffd700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd5d806ace0 0x7fd5d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.176+0000 7fd5eaffd700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd5d806ace0 0x7fd5d806d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc000c00 tx=0x7fd5dc008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:06.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.179+0000 7fd5d3fff700 1 -- 172.21.15.22:0/410438949 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd5e00538d0 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.380+0000 7fd5f18b6700 1 -- 172.21.15.22:0/410438949 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5ec0cb2b0 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.381+0000 7fd5d3fff700 1 -- 172.21.15.22:0/410438949 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7fd5e0056ef0 con 0x7fd5ec0d05e0 2023-07-06T01:20:06.385 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:06.385 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:20:06.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.383+0000 7fd5f18b6700 1 -- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd5d806ace0 msgr2=0x7fd5d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.383+0000 7fd5f18b6700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd5d806ace0 0x7fd5d806d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc000c00 tx=0x7fd5dc008070 comp rx=0 tx=0).stop 2023-07-06T01:20:06.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.383+0000 7fd5f18b6700 1 -- 172.21.15.22:0/410438949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5ec0d05e0 msgr2=0x7fd5ec0c60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.383+0000 7fd5f18b6700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5ec0d05e0 0x7fd5ec0c60a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000c4e0 tx=0x7fd5e000c7f0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.384+0000 7fd5f18b6700 1 -- 172.21.15.22:0/410438949 shutdown_connections 2023-07-06T01:20:06.388 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.384+0000 7fd5f18b6700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd5ec15c950 0x7fd5ec0b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.384+0000 7fd5f18b6700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fd5d806ace0 0x7fd5d806d190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.384+0000 7fd5f18b6700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5ec06b110 0x7fd5ec155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.384+0000 7fd5f18b6700 1 --2- 172.21.15.22:0/410438949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5ec0d05e0 0x7fd5ec0c60a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.384+0000 7fd5f18b6700 1 -- 172.21.15.22:0/410438949 >> 172.21.15.22:0/410438949 conn(0x7fd5ec0999f0 msgr2=0x7fd5ec06e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:06.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.384+0000 7fd5f18b6700 1 -- 172.21.15.22:0/410438949 shutdown_connections 2023-07-06T01:20:06.390 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.384+0000 7fd5f18b6700 1 -- 172.21.15.22:0/410438949 wait complete. 2023-07-06T01:20:06.400 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T01:19:22.873255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T01:19:24.019088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:20:06.401 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:20:06.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.527+0000 7ff604e66700 1 Processor -- start 2023-07-06T01:20:06.530 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.527+0000 7ff604e66700 1 -- start start 2023-07-06T01:20:06.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.528+0000 7ff604e66700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6000ccd70 0x7ff60015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.528+0000 7ff604e66700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff60006d040 0x7ff60006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.528+0000 7ff604e66700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6000cf230 con 0x7ff600159360 2023-07-06T01:20:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.528+0000 7ff604e66700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60015df20 con 0x7ff60006d040 2023-07-06T01:20:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.528+0000 7ff604e66700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000ce620 con 0x7ff6000ccd70 2023-07-06T01:20:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.528+0000 7ff5fe59c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff600159360 0x7ff6000cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37480/0 (socket says 172.21.15.22:37480) 2023-07-06T01:20:06.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.528+0000 7ff5fe59c700 1 -- 172.21.15.22:0/628072897 learned_addr learned my addr 172.21.15.22:0/628072897 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.528+0000 7ff5fdd9b700 1 --2- 172.21.15.22:0/628072897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6000ccd70 0x7ff60015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.528+0000 7ff5fed9d700 1 --2- 172.21.15.22:0/628072897 >> v2:172.21.15.103:3300/0 conn(0x7ff60006d040 0x7ff60006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.528+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258198100 0 0) 0x7ff6000cf230 con 0x7ff600159360 2023-07-06T01:20:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.528+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5e80035b0 con 0x7ff600159360 2023-07-06T01:20:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858174506 0 0) 0x7ff5e80035b0 con 0x7ff600159360 2023-07-06T01:20:06.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6000ccd70 msgr2=0x7ff60015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 --2- 172.21.15.22:0/628072897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6000ccd70 0x7ff60015d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 >> v2:172.21.15.103:3300/0 conn(0x7ff60006d040 msgr2=0x7ff60006d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 --2- 172.21.15.22:0/628072897 >> v2:172.21.15.103:3300/0 conn(0x7ff60006d040 0x7ff60006d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6000ce900 con 0x7ff600159360 2023-07-06T01:20:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7ff5f4001670 con 0x7ff600159360 2023-07-06T01:20:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 >> v1:172.21.15.22:6789/0 conn(0x7ff600159360 legacy=0x7ff6000cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 --2- 172.21.15.22:0/628072897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5e8003a00 0x7ff5e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.542 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 --2- 172.21.15.22:0/628072897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6000ccd70 0x7ff5e8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 --2- 172.21.15.22:0/628072897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e8004d90 0x7ff5e8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60015df20 con 0x7ff5e8004d90 2023-07-06T01:20:06.543 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000ce620 con 0x7ff5e8003a00 2023-07-06T01:20:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e800ba10 con 0x7ff6000ccd70 2023-07-06T01:20:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fed9d700 1 --2- 172.21.15.22:0/628072897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e8004d90 0x7ff5e8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fe59c700 1 --2- 172.21.15.22:0/628072897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5e8003a00 0x7ff5e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fdd9b700 1 --2- 172.21.15.22:0/628072897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6000ccd70 0x7ff5e8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fed9d700 1 -- 172.21.15.22:0/628072897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6000ccd70 msgr2=0x7ff5e8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fed9d700 1 --2- 172.21.15.22:0/628072897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6000ccd70 0x7ff5e8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fed9d700 1 -- 172.21.15.22:0/628072897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5e8003a00 msgr2=0x7ff5e8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:06.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fed9d700 1 --2- 172.21.15.22:0/628072897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5e8003a00 0x7ff5e8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fed9d700 1 -- 172.21.15.22:0/628072897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e80035b0 con 0x7ff5e8004d90 2023-07-06T01:20:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fed9d700 1 --2- 172.21.15.22:0/628072897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e8004d90 0x7ff5e8005200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009fc0 tx=0x7ff5e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c5e3dc02cec1d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800ebd0 con 0x7ff5e8004d90 2023-07-06T01:20:06.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f800f040 con 0x7ff5e8004d90 2023-07-06T01:20:06.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.529+0000 7ff5fd59a700 1 -- 172.21.15.22:0/628072897 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8014480 con 0x7ff5e8004d90 2023-07-06T01:20:06.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.530+0000 7ff604e66700 1 -- 172.21.15.22:0/628072897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e8004d90 msgr2=0x7ff5e8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.530+0000 7ff604e66700 1 --2- 172.21.15.22:0/628072897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5e8004d90 0x7ff5e8005200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009fc0 tx=0x7ff5e8005880 comp rx=0 tx=0).stop 2023-07-06T01:20:06.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.530+0000 7ff5fed9d700 1 -- 172.21.15.22:0/628072897 reap_dead start 2023-07-06T01:20:06.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.530+0000 7ff604e66700 1 -- 172.21.15.22:0/628072897 shutdown_connections 2023-07-06T01:20:06.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.530+0000 7ff604e66700 1 -- 172.21.15.22:0/628072897 >> 172.21.15.22:0/628072897 conn(0x7ff6000b1890 msgr2=0x7ff6000b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:06.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.530+0000 7ff604e66700 1 -- 172.21.15.22:0/628072897 shutdown_connections 2023-07-06T01:20:06.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.530+0000 7ff604e66700 1 -- 172.21.15.22:0/628072897 wait complete. 2023-07-06T01:20:06.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.531+0000 7ff604e66700 1 Processor -- start 2023-07-06T01:20:06.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.532+0000 7ff604e66700 1 -- start start 2023-07-06T01:20:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.532+0000 7ff604e66700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5e8004d90 0x7ff600149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.532+0000 7ff604e66700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60015b310 0x7ff60006dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.532+0000 7ff604e66700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6000b0b70 0x7ff6000b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.532+0000 7ff604e66700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60015c750 con 0x7ff60015b310 2023-07-06T01:20:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.532+0000 7ff604e66700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000b1550 con 0x7ff6000b0b70 2023-07-06T01:20:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.532+0000 7ff604e66700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60015bb40 con 0x7ff5e8004d90 2023-07-06T01:20:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.532+0000 7ff5fdd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60015b310 0x7ff60006dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.532+0000 7ff5fdd9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60015b310 0x7ff60006dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46750/0 (socket says 172.21.15.22:46750) 2023-07-06T01:20:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.532+0000 7ff5fdd9b700 1 -- 172.21.15.22:0/2027326780 learned_addr learned my addr 172.21.15.22:0/2027326780 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.533+0000 7ff5fe59c700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5e8004d90 0x7ff600149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.533+0000 7ff5fed9d700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6000b0b70 0x7ff6000b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.533+0000 7ff5fdd9b700 1 -- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5e8004d90 msgr2=0x7ff600149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.533+0000 7ff5fdd9b700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5e8004d90 0x7ff600149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.533+0000 7ff5fdd9b700 1 -- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6000b0b70 msgr2=0x7ff6000b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.533+0000 7ff5fdd9b700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6000b0b70 0x7ff6000b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.533+0000 7ff5fdd9b700 1 -- 172.21.15.22:0/2027326780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60015be20 con 0x7ff60015b310 2023-07-06T01:20:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.533+0000 7ff5fdd9b700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60015b310 0x7ff60006dbd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0011aa0 tx=0x7ff5f0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.533+0000 7ff5eeffd700 1 -- 172.21.15.22:0/2027326780 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f000f980 con 0x7ff60015b310 2023-07-06T01:20:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.533+0000 7ff5eeffd700 1 -- 172.21.15.22:0/2027326780 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f000fae0 con 0x7ff60015b310 2023-07-06T01:20:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.533+0000 7ff604e66700 1 -- 172.21.15.22:0/2027326780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60015c100 con 0x7ff60015b310 2023-07-06T01:20:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.533+0000 7ff5eeffd700 1 -- 172.21.15.22:0/2027326780 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f000fe90 con 0x7ff60015b310 2023-07-06T01:20:06.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.533+0000 7ff604e66700 1 -- 172.21.15.22:0/2027326780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60015b780 con 0x7ff60015b310 2023-07-06T01:20:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.534+0000 7ff604e66700 1 -- 172.21.15.22:0/2027326780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff600146ca0 con 0x7ff60015b310 2023-07-06T01:20:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.534+0000 7ff5eeffd700 1 -- 172.21.15.22:0/2027326780 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff5f0014440 con 0x7ff60015b310 2023-07-06T01:20:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.535+0000 7ff5eeffd700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff5e006ac70 0x7ff5e006d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.535+0000 7ff5eeffd700 1 -- 172.21.15.22:0/2027326780 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff5f0089cc0 con 0x7ff60015b310 2023-07-06T01:20:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.535+0000 7ff5fe59c700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff5e006ac70 0x7ff5e006d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.536+0000 7ff5fe59c700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff5e006ac70 0x7ff5e006d120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4001eb0 tx=0x7ff5f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:06.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.539+0000 7ff5eeffd700 1 -- 172.21.15.22:0/2027326780 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff600146ca0 con 0x7ff60015b310 2023-07-06T01:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.735+0000 7ff604e66700 1 -- 172.21.15.22:0/2027326780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff600146ca0 con 0x7ff60015b310 2023-07-06T01:20:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.736+0000 7ff5eeffd700 1 -- 172.21.15.22:0/2027326780 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff600146ca0 con 0x7ff60015b310 2023-07-06T01:20:06.739 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:20:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.738+0000 7ff604e66700 1 -- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff5e006ac70 msgr2=0x7ff5e006d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.738+0000 7ff604e66700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff5e006ac70 0x7ff5e006d120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4001eb0 tx=0x7ff5f4009040 comp rx=0 tx=0).stop 2023-07-06T01:20:06.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.738+0000 7ff604e66700 1 -- 172.21.15.22:0/2027326780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60015b310 msgr2=0x7ff60006dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.738+0000 7ff604e66700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60015b310 0x7ff60006dbd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0011aa0 tx=0x7ff5f0011e60 comp rx=0 tx=0).stop 2023-07-06T01:20:06.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.738+0000 7ff604e66700 1 -- 172.21.15.22:0/2027326780 shutdown_connections 2023-07-06T01:20:06.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.738+0000 7ff604e66700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff5e006ac70 0x7ff5e006d120 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.738+0000 7ff604e66700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6000b0b70 0x7ff6000b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.738+0000 7ff604e66700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5e8004d90 0x7ff600149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.738+0000 7ff604e66700 1 --2- 172.21.15.22:0/2027326780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60015b310 0x7ff60006dbd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.738+0000 7ff604e66700 1 -- 172.21.15.22:0/2027326780 >> 172.21.15.22:0/2027326780 conn(0x7ff6000b1890 msgr2=0x7ff6000b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.738+0000 7ff604e66700 1 -- 172.21.15.22:0/2027326780 shutdown_connections 2023-07-06T01:20:06.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.738+0000 7ff604e66700 1 -- 172.21.15.22:0/2027326780 wait complete. 2023-07-06T01:20:06.752 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:20:06.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.877+0000 7f16e362d700 1 Processor -- start 2023-07-06T01:20:06.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.878+0000 7f16e362d700 1 -- start start 2023-07-06T01:20:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.878+0000 7f16e362d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f16dc15c940 0x7f16dc0d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.878+0000 7f16e362d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16dc0cd510 0x7f16dc0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.878+0000 7f16e362d700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f16dc0d0fd0 0x7f16dc1603d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.878+0000 7f16e362d700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16dc09c8a0 con 0x7f16dc15c940 2023-07-06T01:20:06.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.878+0000 7f16e362d700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16dc0d1450 con 0x7f16dc0d0fd0 2023-07-06T01:20:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.878+0000 7f16e362d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc15cd10 con 0x7f16dc0cd510 2023-07-06T01:20:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.878+0000 7f16e13c9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f16dc15c940 0x7f16dc0d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16e13c9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f16dc15c940 0x7f16dc0d2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46756/0 (socket says 172.21.15.22:46756) 2023-07-06T01:20:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16e13c9700 1 -- 172.21.15.22:0/3421461529 learned_addr learned my addr 172.21.15.22:0/3421461529 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16e0bc8700 1 --2- 172.21.15.22:0/3421461529 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16dc0cd510 0x7f16dc0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16e1bca700 1 --2- 172.21.15.22:0/3421461529 >> v2:172.21.15.103:3300/0 conn(0x7f16dc0d0fd0 0x7f16dc1603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16e13c9700 1 -- 172.21.15.22:0/3421461529 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16dc0cd510 msgr2=0x7f16dc0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:06.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16e13c9700 1 --2- 172.21.15.22:0/3421461529 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16dc0cd510 0x7f16dc0d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16e13c9700 1 -- 172.21.15.22:0/3421461529 >> v2:172.21.15.103:3300/0 conn(0x7f16dc0d0fd0 msgr2=0x7f16dc1603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:06.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16e13c9700 1 --2- 172.21.15.22:0/3421461529 >> v2:172.21.15.103:3300/0 conn(0x7f16dc0d0fd0 0x7f16dc1603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16e13c9700 1 -- 172.21.15.22:0/3421461529 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc0ce710 con 0x7f16dc15c940 2023-07-06T01:20:06.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16e1bca700 1 --2- 172.21.15.22:0/3421461529 >> v2:172.21.15.103:3300/0 conn(0x7f16dc0d0fd0 0x7f16dc1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:06.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16e13c9700 1 --2- 172.21.15.22:0/3421461529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16dc15c940 0x7f16dc0d2a10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f16dc08d4a0 tx=0x7f16d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09c8197c3931df server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:06.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16cffff700 1 -- 172.21.15.22:0/3421461529 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d000b870 con 0x7f16dc15c940 2023-07-06T01:20:06.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16cffff700 1 -- 172.21.15.22:0/3421461529 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d000b9d0 con 0x7f16dc15c940 2023-07-06T01:20:06.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.879+0000 7f16cffff700 1 -- 172.21.15.22:0/3421461529 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d000bd60 con 0x7f16dc15c940 2023-07-06T01:20:06.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.880+0000 7f16e362d700 1 -- 172.21.15.22:0/3421461529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16dc15c940 msgr2=0x7f16dc0d2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.880+0000 7f16e362d700 1 --2- 172.21.15.22:0/3421461529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16dc15c940 0x7f16dc0d2a10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f16dc08d4a0 tx=0x7f16d0009bd0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.880+0000 7f16e362d700 1 -- 172.21.15.22:0/3421461529 shutdown_connections 2023-07-06T01:20:06.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.880+0000 7f16e362d700 1 --2- 172.21.15.22:0/3421461529 >> v2:172.21.15.103:3300/0 conn(0x7f16dc0d0fd0 0x7f16dc1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.880+0000 7f16e362d700 1 --2- 172.21.15.22:0/3421461529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16dc15c940 0x7f16dc0d2a10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.880+0000 7f16e362d700 1 --2- 172.21.15.22:0/3421461529 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16dc0cd510 0x7f16dc0d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.880+0000 7f16e362d700 1 -- 172.21.15.22:0/3421461529 >> 172.21.15.22:0/3421461529 conn(0x7f16dc09c2e0 msgr2=0x7f16dc0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:06.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.880+0000 7f16e362d700 1 -- 172.21.15.22:0/3421461529 shutdown_connections 2023-07-06T01:20:06.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.880+0000 7f16e362d700 1 -- 172.21.15.22:0/3421461529 wait complete. 2023-07-06T01:20:06.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.881+0000 7f16e362d700 1 Processor -- start 2023-07-06T01:20:06.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.881+0000 7f16e362d700 1 -- start start 2023-07-06T01:20:06.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e362d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16dc0cd510 0x7f16dc1f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e362d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16dc0d0fd0 0x7f16dc0b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e362d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16dc15c940 0x7f16dc0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e362d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0b5d80 con 0x7f16dc0d0fd0 2023-07-06T01:20:06.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e362d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0b65b0 con 0x7f16dc15c940 2023-07-06T01:20:06.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e362d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0b51f0 con 0x7f16dc0cd510 2023-07-06T01:20:06.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e0bc8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16dc0d0fd0 0x7f16dc0b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e0bc8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16dc0d0fd0 0x7f16dc0b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46764/0 (socket says 172.21.15.22:46764) 2023-07-06T01:20:06.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e0bc8700 1 -- 172.21.15.22:0/1570111322 learned_addr learned my addr 172.21.15.22:0/1570111322 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:06.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e0bc8700 1 -- 172.21.15.22:0/1570111322 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16dc0cd510 msgr2=0x7f16dc1f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e0bc8700 1 --2- 172.21.15.22:0/1570111322 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16dc0cd510 0x7f16dc1f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e0bc8700 1 -- 172.21.15.22:0/1570111322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16dc15c940 msgr2=0x7f16dc0b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:06.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e0bc8700 1 --2- 172.21.15.22:0/1570111322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16dc15c940 0x7f16dc0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:06.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e0bc8700 1 -- 172.21.15.22:0/1570111322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d0009550 con 0x7f16dc0d0fd0 2023-07-06T01:20:06.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.882+0000 7f16e0bc8700 1 --2- 172.21.15.22:0/1570111322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16dc0d0fd0 0x7f16dc0b4770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f16d800c820 tx=0x7f16d800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:06.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.883+0000 7f16cdffb700 1 -- 172.21.15.22:0/1570111322 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8011800 con 0x7f16dc0d0fd0 2023-07-06T01:20:06.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.883+0000 7f16cdffb700 1 -- 172.21.15.22:0/1570111322 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d8011960 con 0x7f16dc0d0fd0 2023-07-06T01:20:06.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.883+0000 7f16e362d700 1 -- 172.21.15.22:0/1570111322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16dc0cca30 con 0x7f16dc0d0fd0 2023-07-06T01:20:06.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.883+0000 7f16cdffb700 1 -- 172.21.15.22:0/1570111322 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8011d10 con 0x7f16dc0d0fd0 2023-07-06T01:20:06.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.883+0000 7f16e362d700 1 -- 172.21.15.22:0/1570111322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16dc0b6240 con 0x7f16dc0d0fd0 2023-07-06T01:20:06.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.884+0000 7f16e362d700 1 -- 172.21.15.22:0/1570111322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16dc0744b0 con 0x7f16dc0d0fd0 2023-07-06T01:20:06.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.884+0000 7f16cdffb700 1 -- 172.21.15.22:0/1570111322 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f16d8018360 con 0x7f16dc0d0fd0 2023-07-06T01:20:06.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.885+0000 7f16cdffb700 1 --2- 172.21.15.22:0/1570111322 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f16c006ace0 0x7f16c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:06.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.885+0000 7f16cdffb700 1 -- 172.21.15.22:0/1570111322 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f16d8089590 con 0x7f16dc0d0fd0 2023-07-06T01:20:06.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.885+0000 7f16e13c9700 1 --2- 172.21.15.22:0/1570111322 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f16c006ace0 0x7f16c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:06.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.885+0000 7f16e13c9700 1 --2- 172.21.15.22:0/1570111322 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f16c006ace0 0x7f16c006d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0b0a00 tx=0x7f16d0010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:06.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:06.889+0000 7f16cdffb700 1 -- 172.21.15.22:0/1570111322 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f16d8054510 con 0x7f16dc0d0fd0 2023-07-06T01:20:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.086+0000 7f16e362d700 1 -- 172.21.15.22:0/1570111322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16dc0b66f0 con 0x7f16dc0d0fd0 2023-07-06T01:20:07.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.087+0000 7f16cdffb700 1 -- 172.21.15.22:0/1570111322 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16d801f020 con 0x7f16dc0d0fd0 2023-07-06T01:20:07.090 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:20:07.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.088+0000 7f16e362d700 1 -- 172.21.15.22:0/1570111322 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f16c006ace0 msgr2=0x7f16c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.088+0000 7f16e362d700 1 --2- 172.21.15.22:0/1570111322 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f16c006ace0 0x7f16c006d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0b0a00 tx=0x7f16d0010000 comp rx=0 tx=0).stop 2023-07-06T01:20:07.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.088+0000 7f16e362d700 1 -- 172.21.15.22:0/1570111322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16dc0d0fd0 msgr2=0x7f16dc0b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.088+0000 7f16e362d700 1 --2- 172.21.15.22:0/1570111322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16dc0d0fd0 0x7f16dc0b4770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f16d800c820 tx=0x7f16d800cbe0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.089+0000 7f16e362d700 1 -- 172.21.15.22:0/1570111322 shutdown_connections 2023-07-06T01:20:07.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.089+0000 7f16e362d700 1 --2- 172.21.15.22:0/1570111322 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f16c006ace0 0x7f16c006d190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.089+0000 7f16e362d700 1 --2- 172.21.15.22:0/1570111322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16dc15c940 0x7f16dc0b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.089+0000 7f16e362d700 1 --2- 172.21.15.22:0/1570111322 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f16dc0cd510 0x7f16dc1f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.089+0000 7f16e362d700 1 --2- 172.21.15.22:0/1570111322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f16dc0d0fd0 0x7f16dc0b4770 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.089+0000 7f16e362d700 1 -- 172.21.15.22:0/1570111322 >> 172.21.15.22:0/1570111322 conn(0x7f16dc09c2e0 msgr2=0x7f16dc15b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:07.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.089+0000 7f16e362d700 1 -- 172.21.15.22:0/1570111322 shutdown_connections 2023-07-06T01:20:07.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.089+0000 7f16e362d700 1 -- 172.21.15.22:0/1570111322 wait complete. 2023-07-06T01:20:07.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T01:20:07.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.228+0000 7ff0c633d700 1 Processor -- start 2023-07-06T01:20:07.231 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.228+0000 7ff0c633d700 1 -- start start 2023-07-06T01:20:07.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.229+0000 7ff0c633d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0c006a710 0x7ff0c00ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.229+0000 7ff0c633d700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff0c00cf090 0x7ff0c0160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.229+0000 7ff0c633d700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c00ca2b0 con 0x7ff0c015cb50 2023-07-06T01:20:07.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.229+0000 7ff0c633d700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c00af440 con 0x7ff0c00cf090 2023-07-06T01:20:07.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.229+0000 7ff0c633d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c00cfc90 con 0x7ff0c006a710 2023-07-06T01:20:07.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0c48da700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff0c00cf090 0x7ff0c0160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0bf7fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0c006a710 0x7ff0c00ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0bffff700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7ff0c015cb50 0x7ff0c006a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:39578/0 (socket says 172.21.15.22:39578) 2023-07-06T01:20:07.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0bffff700 1 -- 172.21.15.22:0/2539475240 learned_addr learned my addr 172.21.15.22:0/2539475240 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:07.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0beffd700 1 -- 172.21.15.22:0/2539475240 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941599754 0 0) 0x7ff0c00ca2b0 con 0x7ff0c015cb50 2023-07-06T01:20:07.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0beffd700 1 -- 172.21.15.22:0/2539475240 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ac0035b0 con 0x7ff0c015cb50 2023-07-06T01:20:07.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0c48da700 1 -- 172.21.15.22:0/2539475240 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0c006a710 msgr2=0x7ff0c00ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:07.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0c48da700 1 --2- 172.21.15.22:0/2539475240 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0c006a710 0x7ff0c00ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0c48da700 1 -- 172.21.15.22:0/2539475240 >> v1:172.21.15.103:6789/0 conn(0x7ff0c015cb50 legacy=0x7ff0c006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0c48da700 1 -- 172.21.15.22:0/2539475240 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c00cff30 con 0x7ff0c00cf090 2023-07-06T01:20:07.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0bf7fe700 1 --2- 172.21.15.22:0/2539475240 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0c006a710 0x7ff0c00ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:07.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0c48da700 1 --2- 172.21.15.22:0/2539475240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0c00cf090 0x7ff0c0160710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8009d90 tx=0x7ff0b800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=635d74258b747182 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:07.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0beffd700 1 -- 172.21.15.22:0/2539475240 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b800e070 con 0x7ff0c00cf090 2023-07-06T01:20:07.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0beffd700 1 -- 172.21.15.22:0/2539475240 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b800bb20 con 0x7ff0c00cf090 2023-07-06T01:20:07.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.230+0000 7ff0beffd700 1 -- 172.21.15.22:0/2539475240 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8012430 con 0x7ff0c00cf090 2023-07-06T01:20:07.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.231+0000 7ff0c633d700 1 -- 172.21.15.22:0/2539475240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0c00cf090 msgr2=0x7ff0c0160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.231+0000 7ff0c633d700 1 --2- 172.21.15.22:0/2539475240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0c00cf090 0x7ff0c0160710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8009d90 tx=0x7ff0b800b330 comp rx=0 tx=0).stop 2023-07-06T01:20:07.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.231+0000 7ff0c633d700 1 -- 172.21.15.22:0/2539475240 shutdown_connections 2023-07-06T01:20:07.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.231+0000 7ff0c633d700 1 --2- 172.21.15.22:0/2539475240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0c00cf090 0x7ff0c0160710 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.231+0000 7ff0c633d700 1 --2- 172.21.15.22:0/2539475240 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0c006a710 0x7ff0c00ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.231+0000 7ff0c633d700 1 -- 172.21.15.22:0/2539475240 >> 172.21.15.22:0/2539475240 conn(0x7ff0c006b110 msgr2=0x7ff0c006b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:07.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.231+0000 7ff0c633d700 1 -- 172.21.15.22:0/2539475240 shutdown_connections 2023-07-06T01:20:07.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.231+0000 7ff0c633d700 1 -- 172.21.15.22:0/2539475240 wait complete. 2023-07-06T01:20:07.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.232+0000 7ff0c633d700 1 Processor -- start 2023-07-06T01:20:07.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.232+0000 7ff0c633d700 1 -- start start 2023-07-06T01:20:07.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.233+0000 7ff0c633d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0c006a710 0x7ff0c0149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.233+0000 7ff0c633d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0c00cf090 0x7ff0c015e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.233+0000 7ff0c633d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015cb50 0x7ff0c01f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.233+0000 7ff0c633d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c015ff50 con 0x7ff0c015cb50 2023-07-06T01:20:07.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.233+0000 7ff0c633d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c0149660 con 0x7ff0c006a710 2023-07-06T01:20:07.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.233+0000 7ff0c633d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c015f990 con 0x7ff0c00cf090 2023-07-06T01:20:07.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.233+0000 7ff0c48da700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015cb50 0x7ff0c01f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.233+0000 7ff0c48da700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015cb50 0x7ff0c01f5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46780/0 (socket says 172.21.15.22:46780) 2023-07-06T01:20:07.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.233+0000 7ff0c48da700 1 -- 172.21.15.22:0/1921572639 learned_addr learned my addr 172.21.15.22:0/1921572639 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:07.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.233+0000 7ff0bffff700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0c006a710 0x7ff0c0149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.233+0000 7ff0bf7fe700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0c00cf090 0x7ff0c015e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.234+0000 7ff0c48da700 1 -- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0c00cf090 msgr2=0x7ff0c015e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.234+0000 7ff0c48da700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0c00cf090 0x7ff0c015e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.234+0000 7ff0c48da700 1 -- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0c006a710 msgr2=0x7ff0c0149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.234+0000 7ff0c48da700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0c006a710 0x7ff0c0149120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.234+0000 7ff0c48da700 1 -- 172.21.15.22:0/1921572639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a80094a0 con 0x7ff0c015cb50 2023-07-06T01:20:07.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.234+0000 7ff0c48da700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015cb50 0x7ff0c01f5de0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8014040 tx=0x7ff0b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:07.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.234+0000 7ff0bcff9700 1 -- 172.21.15.22:0/1921572639 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b80129f0 con 0x7ff0c015cb50 2023-07-06T01:20:07.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.234+0000 7ff0bcff9700 1 -- 172.21.15.22:0/1921572639 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b8020070 con 0x7ff0c015cb50 2023-07-06T01:20:07.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.234+0000 7ff0c633d700 1 -- 172.21.15.22:0/1921572639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b8009a70 con 0x7ff0c015cb50 2023-07-06T01:20:07.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.234+0000 7ff0bcff9700 1 -- 172.21.15.22:0/1921572639 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8012dd0 con 0x7ff0c015cb50 2023-07-06T01:20:07.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.234+0000 7ff0c633d700 1 -- 172.21.15.22:0/1921572639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c015dad0 con 0x7ff0c015cb50 2023-07-06T01:20:07.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.235+0000 7ff0c633d700 1 -- 172.21.15.22:0/1921572639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c015d930 con 0x7ff0c015cb50 2023-07-06T01:20:07.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.235+0000 7ff0bcff9700 1 -- 172.21.15.22:0/1921572639 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff0b8004040 con 0x7ff0c015cb50 2023-07-06T01:20:07.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.236+0000 7ff0bcff9700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff0a006ace0 0x7ff0a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.236+0000 7ff0bcff9700 1 -- 172.21.15.22:0/1921572639 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff0b8022070 con 0x7ff0c015cb50 2023-07-06T01:20:07.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.236+0000 7ff0bffff700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff0a006ace0 0x7ff0a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.236+0000 7ff0bffff700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff0a006ace0 0x7ff0a006d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0007fd0 tx=0x7ff0b0007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:07.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.240+0000 7ff0bcff9700 1 -- 172.21.15.22:0/1921572639 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff0c015d930 con 0x7ff0c015cb50 2023-07-06T01:20:07.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.436+0000 7ff0c633d700 1 -- 172.21.15.22:0/1921572639 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff0c015d930 con 0x7ff0c015cb50 2023-07-06T01:20:07.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.436+0000 7ff0bcff9700 1 -- 172.21.15.22:0/1921572639 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff0c015d930 con 0x7ff0c015cb50 2023-07-06T01:20:07.439 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-07-06T01:20:07.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.438+0000 7ff0c633d700 1 -- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff0a006ace0 msgr2=0x7ff0a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.438+0000 7ff0c633d700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff0a006ace0 0x7ff0a006d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0007fd0 tx=0x7ff0b0007dc0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.438+0000 7ff0c633d700 1 -- 172.21.15.22:0/1921572639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015cb50 msgr2=0x7ff0c01f5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.438+0000 7ff0c633d700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015cb50 0x7ff0c01f5de0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8014040 tx=0x7ff0b8004ad0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.438+0000 7ff0c633d700 1 -- 172.21.15.22:0/1921572639 shutdown_connections 2023-07-06T01:20:07.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.438+0000 7ff0c633d700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0c015cb50 0x7ff0c01f5de0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.438+0000 7ff0c633d700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff0a006ace0 0x7ff0a006d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.438+0000 7ff0c633d700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0c006a710 0x7ff0c0149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.438+0000 7ff0c633d700 1 --2- 172.21.15.22:0/1921572639 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0c00cf090 0x7ff0c015e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.438+0000 7ff0c633d700 1 -- 172.21.15.22:0/1921572639 >> 172.21.15.22:0/1921572639 conn(0x7ff0c006b110 msgr2=0x7ff0c00ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:07.442 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.438+0000 7ff0c633d700 1 -- 172.21.15.22:0/1921572639 shutdown_connections 2023-07-06T01:20:07.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.438+0000 7ff0c633d700 1 -- 172.21.15.22:0/1921572639 wait complete. 2023-07-06T01:20:07.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T01:20:07.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.578+0000 7ff0554e1700 1 Processor -- start 2023-07-06T01:20:07.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.579+0000 7ff0554e1700 1 -- start start 2023-07-06T01:20:07.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.579+0000 7ff0554e1700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff0500b36f0 0x7ff0500cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.579+0000 7ff0554e1700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0500cf5a0 0x7ff0500d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.579+0000 7ff0554e1700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0500cebd0 con 0x7ff0500b3b70 2023-07-06T01:20:07.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.579+0000 7ff0554e1700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05015cff0 con 0x7ff0500b36f0 2023-07-06T01:20:07.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.579+0000 7ff0554e1700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500cdfc0 con 0x7ff0500cf5a0 2023-07-06T01:20:07.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.579+0000 7ff04effd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff0500b36f0 0x7ff0500cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.579+0000 7ff04f7fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff0500b3b70 0x7ff0500b5da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37532/0 (socket says 172.21.15.22:37532) 2023-07-06T01:20:07.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.580+0000 7ff04f7fe700 1 -- 172.21.15.22:0/1277781080 learned_addr learned my addr 172.21.15.22:0/1277781080 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:07.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.580+0000 7ff04e7fc700 1 --2- 172.21.15.22:0/1277781080 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0500cf5a0 0x7ff0500d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.580+0000 7ff04effd700 1 -- 172.21.15.22:0/1277781080 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0500cf5a0 msgr2=0x7ff0500d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:07.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.580+0000 7ff04effd700 1 --2- 172.21.15.22:0/1277781080 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0500cf5a0 0x7ff0500d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.580+0000 7ff04effd700 1 -- 172.21.15.22:0/1277781080 >> v1:172.21.15.22:6789/0 conn(0x7ff0500b3b70 legacy=0x7ff0500b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.580+0000 7ff04effd700 1 -- 172.21.15.22:0/1277781080 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0500ce2a0 con 0x7ff0500b36f0 2023-07-06T01:20:07.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.580+0000 7ff04dffb700 1 -- 172.21.15.22:0/1277781080 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795520184 0 0) 0x7ff0500cebd0 con 0x7ff0500b3b70 2023-07-06T01:20:07.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.580+0000 7ff04e7fc700 1 --2- 172.21.15.22:0/1277781080 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0500cf5a0 0x7ff0500d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:07.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.580+0000 7ff04effd700 1 --2- 172.21.15.22:0/1277781080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0500b36f0 0x7ff0500cf060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff044009840 tx=0x7ff044009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3b64f9d7d98b17 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:07.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.580+0000 7ff04dffb700 1 -- 172.21.15.22:0/1277781080 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044011830 con 0x7ff0500b36f0 2023-07-06T01:20:07.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.580+0000 7ff04dffb700 1 -- 172.21.15.22:0/1277781080 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff044011990 con 0x7ff0500b36f0 2023-07-06T01:20:07.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.580+0000 7ff04dffb700 1 -- 172.21.15.22:0/1277781080 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044011d20 con 0x7ff0500b36f0 2023-07-06T01:20:07.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.581+0000 7ff0554e1700 1 -- 172.21.15.22:0/1277781080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0500b36f0 msgr2=0x7ff0500cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.581+0000 7ff0554e1700 1 --2- 172.21.15.22:0/1277781080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0500b36f0 0x7ff0500cf060 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff044009840 tx=0x7ff044009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:07.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.581+0000 7ff0554e1700 1 -- 172.21.15.22:0/1277781080 shutdown_connections 2023-07-06T01:20:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.581+0000 7ff0554e1700 1 --2- 172.21.15.22:0/1277781080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0500b36f0 0x7ff0500cf060 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.581+0000 7ff0554e1700 1 --2- 172.21.15.22:0/1277781080 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0500cf5a0 0x7ff0500d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.581+0000 7ff0554e1700 1 -- 172.21.15.22:0/1277781080 >> 172.21.15.22:0/1277781080 conn(0x7ff05009c2e0 msgr2=0x7ff0500d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.581+0000 7ff0554e1700 1 -- 172.21.15.22:0/1277781080 shutdown_connections 2023-07-06T01:20:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.581+0000 7ff0554e1700 1 -- 172.21.15.22:0/1277781080 wait complete. 2023-07-06T01:20:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.582+0000 7ff0554e1700 1 Processor -- start 2023-07-06T01:20:07.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.582+0000 7ff0554e1700 1 -- start start 2023-07-06T01:20:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff0554e1700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0500b36f0 0x7ff0501f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff0554e1700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0500b3b70 0x7ff0501470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff0554e1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0500cf5a0 0x7ff05015b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff0554e1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500cc2a0 con 0x7ff0500cf5a0 2023-07-06T01:20:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff0554e1700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0501f6040 con 0x7ff0500b36f0 2023-07-06T01:20:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff0554e1700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0501f6320 con 0x7ff0500b3b70 2023-07-06T01:20:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0500cf5a0 0x7ff05015b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0500cf5a0 0x7ff05015b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46794/0 (socket says 172.21.15.22:46794) 2023-07-06T01:20:07.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff04f7fe700 1 -- 172.21.15.22:0/2345190241 learned_addr learned my addr 172.21.15.22:0/2345190241 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:07.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff04effd700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0500b36f0 0x7ff0501f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff04e7fc700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0500b3b70 0x7ff0501470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff04f7fe700 1 -- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0500b3b70 msgr2=0x7ff0501470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff04f7fe700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0500b3b70 0x7ff0501470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff04f7fe700 1 -- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0500b36f0 msgr2=0x7ff0501f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.583+0000 7ff04f7fe700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0500b36f0 0x7ff0501f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.584+0000 7ff04f7fe700 1 -- 172.21.15.22:0/2345190241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0440094a0 con 0x7ff0500cf5a0 2023-07-06T01:20:07.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.584+0000 7ff04f7fe700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0500cf5a0 0x7ff05015b830 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff048002230 tx=0x7ff04800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:07.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.584+0000 7ff0377fe700 1 -- 172.21.15.22:0/2345190241 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04800aac0 con 0x7ff0500cf5a0 2023-07-06T01:20:07.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.584+0000 7ff0377fe700 1 -- 172.21.15.22:0/2345190241 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04800ac40 con 0x7ff0500cf5a0 2023-07-06T01:20:07.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.584+0000 7ff0377fe700 1 -- 172.21.15.22:0/2345190241 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0480070d0 con 0x7ff0500cf5a0 2023-07-06T01:20:07.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.584+0000 7ff0554e1700 1 -- 172.21.15.22:0/2345190241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0501f6600 con 0x7ff0500cf5a0 2023-07-06T01:20:07.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.584+0000 7ff0554e1700 1 -- 172.21.15.22:0/2345190241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0501f5c80 con 0x7ff0500cf5a0 2023-07-06T01:20:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.585+0000 7ff0554e1700 1 -- 172.21.15.22:0/2345190241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0500b4840 con 0x7ff0500cf5a0 2023-07-06T01:20:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.586+0000 7ff0377fe700 1 -- 172.21.15.22:0/2345190241 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff048007250 con 0x7ff0500cf5a0 2023-07-06T01:20:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.586+0000 7ff0377fe700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff03c06ace0 0x7ff03c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.586+0000 7ff0377fe700 1 -- 172.21.15.22:0/2345190241 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff048088e30 con 0x7ff0500cf5a0 2023-07-06T01:20:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.586+0000 7ff04effd700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff03c06ace0 0x7ff03c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.587+0000 7ff04effd700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff03c06ace0 0x7ff03c06d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff0440090b0 tx=0x7ff044008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:07.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.591+0000 7ff0377fe700 1 -- 172.21.15.22:0/2345190241 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff048053f70 con 0x7ff0500cf5a0 2023-07-06T01:20:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.787+0000 7ff0554e1700 1 -- 172.21.15.22:0/2345190241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0500744b0 con 0x7ff0500cf5a0 2023-07-06T01:20:07.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.787+0000 7ff0377fe700 1 -- 172.21.15.22:0/2345190241 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0480180a0 con 0x7ff0500cf5a0 2023-07-06T01:20:07.791 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-07-06T01:20:07.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.789+0000 7ff0554e1700 1 -- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff03c06ace0 msgr2=0x7ff03c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.789+0000 7ff0554e1700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff03c06ace0 0x7ff03c06d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff0440090b0 tx=0x7ff044008030 comp rx=0 tx=0).stop 2023-07-06T01:20:07.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.789+0000 7ff0554e1700 1 -- 172.21.15.22:0/2345190241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0500cf5a0 msgr2=0x7ff05015b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.789+0000 7ff0554e1700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0500cf5a0 0x7ff05015b830 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff048002230 tx=0x7ff04800cf50 comp rx=0 tx=0).stop 2023-07-06T01:20:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.789+0000 7ff0554e1700 1 -- 172.21.15.22:0/2345190241 shutdown_connections 2023-07-06T01:20:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.789+0000 7ff0554e1700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0500cf5a0 0x7ff05015b830 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.789+0000 7ff0554e1700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff03c06ace0 0x7ff03c06d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.789+0000 7ff0554e1700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff0500b36f0 0x7ff0501f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.789+0000 7ff0554e1700 1 --2- 172.21.15.22:0/2345190241 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff0500b3b70 0x7ff0501470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.789+0000 7ff0554e1700 1 -- 172.21.15.22:0/2345190241 >> 172.21.15.22:0/2345190241 conn(0x7ff05009c2e0 msgr2=0x7ff05006abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.789+0000 7ff0554e1700 1 -- 172.21.15.22:0/2345190241 shutdown_connections 2023-07-06T01:20:07.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.789+0000 7ff0554e1700 1 -- 172.21.15.22:0/2345190241 wait complete. 2023-07-06T01:20:07.805 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T01:20:07.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.930+0000 7f552a533700 1 Processor -- start 2023-07-06T01:20:07.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.930+0000 7f552a533700 1 -- start start 2023-07-06T01:20:07.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f552a533700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f552415a690 0x7f55240ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f552a533700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f55240cee60 0x7f552406a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f552a533700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55240cd640 con 0x7f55240c9a40 2023-07-06T01:20:07.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f552a533700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55240cd920 con 0x7f55240cee60 2023-07-06T01:20:07.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f552a533700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55240cdc00 con 0x7f552415a690 2023-07-06T01:20:07.934 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f5528ad0700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f55240cee60 0x7f552406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f5523fff700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f55240c9a40 0x7f55240c9e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:39606/0 (socket says 172.21.15.22:39606) 2023-07-06T01:20:07.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f5523fff700 1 -- 172.21.15.22:0/2215505155 learned_addr learned my addr 172.21.15.22:0/2215505155 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:07.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f55237fe700 1 --2- 172.21.15.22:0/2215505155 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f552415a690 0x7f55240ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f5528ad0700 1 -- 172.21.15.22:0/2215505155 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f552415a690 msgr2=0x7f55240ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:07.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f5528ad0700 1 --2- 172.21.15.22:0/2215505155 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f552415a690 0x7f55240ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f5528ad0700 1 -- 172.21.15.22:0/2215505155 >> v1:172.21.15.103:6789/0 conn(0x7f55240c9a40 legacy=0x7f55240c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f5528ad0700 1 -- 172.21.15.22:0/2215505155 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55240cdee0 con 0x7f55240cee60 2023-07-06T01:20:07.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f5528ad0700 1 --2- 172.21.15.22:0/2215505155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55240cee60 0x7f552406a000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f551c009d90 tx=0x7f551c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d60895fe44a01b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:07.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f5522ffd700 1 -- 172.21.15.22:0/2215505155 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c00e070 con 0x7f55240cee60 2023-07-06T01:20:07.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f5522ffd700 1 -- 172.21.15.22:0/2215505155 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c00bb20 con 0x7f55240cee60 2023-07-06T01:20:07.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.931+0000 7f5522ffd700 1 -- 172.21.15.22:0/2215505155 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c012430 con 0x7f55240cee60 2023-07-06T01:20:07.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.932+0000 7f552a533700 1 -- 172.21.15.22:0/2215505155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55240cee60 msgr2=0x7f552406a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.932+0000 7f552a533700 1 --2- 172.21.15.22:0/2215505155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55240cee60 0x7f552406a000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f551c009d90 tx=0x7f551c00b330 comp rx=0 tx=0).stop 2023-07-06T01:20:07.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.932+0000 7f552a533700 1 -- 172.21.15.22:0/2215505155 shutdown_connections 2023-07-06T01:20:07.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.932+0000 7f552a533700 1 --2- 172.21.15.22:0/2215505155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55240cee60 0x7f552406a000 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.932+0000 7f552a533700 1 --2- 172.21.15.22:0/2215505155 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f552415a690 0x7f55240ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.932+0000 7f552a533700 1 -- 172.21.15.22:0/2215505155 >> 172.21.15.22:0/2215505155 conn(0x7f55240ccb00 msgr2=0x7f55240cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:07.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.932+0000 7f552a533700 1 -- 172.21.15.22:0/2215505155 shutdown_connections 2023-07-06T01:20:07.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.932+0000 7f552a533700 1 -- 172.21.15.22:0/2215505155 wait complete. 2023-07-06T01:20:07.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.933+0000 7f552a533700 1 Processor -- start 2023-07-06T01:20:07.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.933+0000 7f552a533700 1 -- start start 2023-07-06T01:20:07.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f552a533700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55240c9a40 0x7f55240c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f552a533700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55240cee60 0x7f55240b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f552a533700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f552415a690 0x7f552415d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f552a533700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55240c7f70 con 0x7f55240c9a40 2023-07-06T01:20:07.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f552a533700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55240c7360 con 0x7f55240cee60 2023-07-06T01:20:07.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f552a533700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55240c7640 con 0x7f552415a690 2023-07-06T01:20:07.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f5523fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55240c9a40 0x7f55240c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f5523fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55240c9a40 0x7f55240c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46806/0 (socket says 172.21.15.22:46806) 2023-07-06T01:20:07.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f5523fff700 1 -- 172.21.15.22:0/4203315679 learned_addr learned my addr 172.21.15.22:0/4203315679 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:07.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f55237fe700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55240cee60 0x7f55240b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f5528ad0700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f552415a690 0x7f552415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f5523fff700 1 -- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f552415a690 msgr2=0x7f552415d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f5523fff700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f552415a690 0x7f552415d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f5523fff700 1 -- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55240cee60 msgr2=0x7f55240b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:07.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f5523fff700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55240cee60 0x7f55240b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:07.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f5523fff700 1 -- 172.21.15.22:0/4203315679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c009a70 con 0x7f55240c9a40 2023-07-06T01:20:07.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.934+0000 7f55237fe700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55240cee60 0x7f55240b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:07.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.935+0000 7f5523fff700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55240c9a40 0x7f55240c8400 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f551400c520 tx=0x7f551400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:07.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.935+0000 7f5520ff9700 1 -- 172.21.15.22:0/4203315679 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514005d70 con 0x7f55240c9a40 2023-07-06T01:20:07.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.935+0000 7f5520ff9700 1 -- 172.21.15.22:0/4203315679 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55140070d0 con 0x7f55240c9a40 2023-07-06T01:20:07.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.935+0000 7f552a533700 1 -- 172.21.15.22:0/4203315679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55240c7920 con 0x7f55240c9a40 2023-07-06T01:20:07.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.935+0000 7f5520ff9700 1 -- 172.21.15.22:0/4203315679 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514007440 con 0x7f55240c9a40 2023-07-06T01:20:07.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.935+0000 7f552a533700 1 -- 172.21.15.22:0/4203315679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55240b1610 con 0x7f55240c9a40 2023-07-06T01:20:07.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.936+0000 7f552a533700 1 -- 172.21.15.22:0/4203315679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55240b17a0 con 0x7f55240c9a40 2023-07-06T01:20:07.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.936+0000 7f5520ff9700 1 -- 172.21.15.22:0/4203315679 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f55140075a0 con 0x7f55240c9a40 2023-07-06T01:20:07.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.937+0000 7f5520ff9700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f551006ace0 0x7f551006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:07.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.937+0000 7f5520ff9700 1 -- 172.21.15.22:0/4203315679 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5514088890 con 0x7f55240c9a40 2023-07-06T01:20:07.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.937+0000 7f55237fe700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f551006ace0 0x7f551006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:07.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.937+0000 7f55237fe700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f551006ace0 0x7f551006d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f55240734c0 tx=0x7f5518008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:07.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:07.942+0000 7f5520ff9700 1 -- 172.21.15.22:0/4203315679 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f55140539f0 con 0x7f55240c9a40 2023-07-06T01:20:08.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.153+0000 7f552a533700 1 -- 172.21.15.22:0/4203315679 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f55240744b0 con 0x7f55240c9a40 2023-07-06T01:20:08.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.153+0000 7f5520ff9700 1 -- 172.21.15.22:0/4203315679 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5514057010 con 0x7f55240c9a40 2023-07-06T01:20:08.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.155+0000 7f552a533700 1 -- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f551006ace0 msgr2=0x7f551006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:08.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.155+0000 7f552a533700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f551006ace0 0x7f551006d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f55240734c0 tx=0x7f5518008040 comp rx=0 tx=0).stop 2023-07-06T01:20:08.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.155+0000 7f552a533700 1 -- 172.21.15.22:0/4203315679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55240c9a40 msgr2=0x7f55240c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:08.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.155+0000 7f552a533700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55240c9a40 0x7f55240c8400 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f551400c520 tx=0x7f551400c8e0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.155+0000 7f552a533700 1 -- 172.21.15.22:0/4203315679 shutdown_connections 2023-07-06T01:20:08.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.155+0000 7f552a533700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f552415a690 0x7f552415d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.155+0000 7f552a533700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55240c9a40 0x7f55240c8400 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.155+0000 7f552a533700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f551006ace0 0x7f551006d190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.155+0000 7f552a533700 1 --2- 172.21.15.22:0/4203315679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55240cee60 0x7f55240b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.155+0000 7f552a533700 1 -- 172.21.15.22:0/4203315679 >> 172.21.15.22:0/4203315679 conn(0x7f55240ccb00 msgr2=0x7f5524065d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:08.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.155+0000 7f552a533700 1 -- 172.21.15.22:0/4203315679 shutdown_connections 2023-07-06T01:20:08.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.155+0000 7f552a533700 1 -- 172.21.15.22:0/4203315679 wait complete. 2023-07-06T01:20:08.159 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-07-06T01:20:08.169 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-07-06T01:20:08.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:08.171 DEBUG:teuthology.orchestra.run.smithi103:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-07-06T01:20:08.212 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi103.front.sepia.ceph.com 2023-07-06T01:20:08.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:08.213 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T01:20:08.271 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T01:20:08.271 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:20:08.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.394+0000 7f20058c8700 1 Processor -- start 2023-07-06T01:20:08.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.394+0000 7f20058c8700 1 -- start start 2023-07-06T01:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.395+0000 7f20058c8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2000159360 0x7f200015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.395+0000 7f20058c8700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f20000cc660 0x7f20000ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.395+0000 7f20058c8700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20000cf230 con 0x7f2000070a20 2023-07-06T01:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.395+0000 7f20058c8700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f200015df20 con 0x7f2000159360 2023-07-06T01:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.395+0000 7f20058c8700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20000ccff0 con 0x7f20000cc660 2023-07-06T01:20:08.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.395+0000 7f1ffeffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2000159360 0x7f200015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.395+0000 7f1fff7fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f2000070a20 0x7f2000070e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37560/0 (socket says 172.21.15.22:37560) 2023-07-06T01:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.395+0000 7f1fff7fe700 1 -- 172.21.15.22:0/4256224917 learned_addr learned my addr 172.21.15.22:0/4256224917 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.395+0000 7f1ffe7fc700 1 --2- 172.21.15.22:0/4256224917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f20000cc660 0x7f20000ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.395+0000 7f1ffeffd700 1 -- 172.21.15.22:0/4256224917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f20000cc660 msgr2=0x7f20000ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f1ffeffd700 1 --2- 172.21.15.22:0/4256224917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f20000cc660 0x7f20000ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f1ffeffd700 1 -- 172.21.15.22:0/4256224917 >> v1:172.21.15.22:6789/0 conn(0x7f2000070a20 legacy=0x7f2000070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:08.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f1ffeffd700 1 -- 172.21.15.22:0/4256224917 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20000ce620 con 0x7f2000159360 2023-07-06T01:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f1ffdffb700 1 -- 172.21.15.22:0/4256224917 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912031246 0 0) 0x7f20000cf230 con 0x7f2000070a20 2023-07-06T01:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f1ffeffd700 1 --2- 172.21.15.22:0/4256224917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2000159360 0x7f200015d9e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009840 tx=0x7f1ff4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3114c91dd324132 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f1ffdffb700 1 -- 172.21.15.22:0/4256224917 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4011830 con 0x7f2000159360 2023-07-06T01:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f1ffdffb700 1 -- 172.21.15.22:0/4256224917 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff4011990 con 0x7f2000159360 2023-07-06T01:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f1ffdffb700 1 -- 172.21.15.22:0/4256224917 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4011d20 con 0x7f2000159360 2023-07-06T01:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f20058c8700 1 -- 172.21.15.22:0/4256224917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2000159360 msgr2=0x7f200015d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:08.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f20058c8700 1 --2- 172.21.15.22:0/4256224917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2000159360 0x7f200015d9e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009840 tx=0x7f1ff4009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f20058c8700 1 -- 172.21.15.22:0/4256224917 shutdown_connections 2023-07-06T01:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f20058c8700 1 --2- 172.21.15.22:0/4256224917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2000159360 0x7f200015d9e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f20058c8700 1 --2- 172.21.15.22:0/4256224917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f20000cc660 0x7f20000ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.396+0000 7f20058c8700 1 -- 172.21.15.22:0/4256224917 >> 172.21.15.22:0/4256224917 conn(0x7f20000b1890 msgr2=0x7f20000b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.397+0000 7f20058c8700 1 -- 172.21.15.22:0/4256224917 shutdown_connections 2023-07-06T01:20:08.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.397+0000 7f20058c8700 1 -- 172.21.15.22:0/4256224917 wait complete. 2023-07-06T01:20:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.398+0000 7f20058c8700 1 Processor -- start 2023-07-06T01:20:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.398+0000 7f20058c8700 1 -- start start 2023-07-06T01:20:08.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f20058c8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2000070a20 0x7f200015bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f20058c8700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f20000cc660 0x7f20000b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f20058c8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2000159360 0x7f200015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f20058c8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200015c670 con 0x7f2000159360 2023-07-06T01:20:08.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f20058c8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20000c96f0 con 0x7f2000070a20 2023-07-06T01:20:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f20058c8700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200015c390 con 0x7f20000cc660 2023-07-06T01:20:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1fff7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2000159360 0x7f200015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1fff7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2000159360 0x7f200015a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46822/0 (socket says 172.21.15.22:46822) 2023-07-06T01:20:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1fff7fe700 1 -- 172.21.15.22:0/929119563 learned_addr learned my addr 172.21.15.22:0/929119563 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1ffeffd700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2000070a20 0x7f200015bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1ffe7fc700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f20000cc660 0x7f20000b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:08.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1fff7fe700 1 -- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f20000cc660 msgr2=0x7f20000b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1fff7fe700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f20000cc660 0x7f20000b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1fff7fe700 1 -- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2000070a20 msgr2=0x7f200015bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1fff7fe700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2000070a20 0x7f200015bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1fff7fe700 1 -- 172.21.15.22:0/929119563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff40094a0 con 0x7f2000159360 2023-07-06T01:20:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1fff7fe700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2000159360 0x7f200015a1f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800cca0 tx=0x7f1ff800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1fe7fff700 1 -- 172.21.15.22:0/929119563 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800ab70 con 0x7f2000159360 2023-07-06T01:20:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1fe7fff700 1 -- 172.21.15.22:0/929119563 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff800acd0 con 0x7f2000159360 2023-07-06T01:20:08.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f20058c8700 1 -- 172.21.15.22:0/929119563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200006d1a0 con 0x7f2000159360 2023-07-06T01:20:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f1fe7fff700 1 -- 172.21.15.22:0/929119563 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8007310 con 0x7f2000159360 2023-07-06T01:20:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.399+0000 7f20058c8700 1 -- 172.21.15.22:0/929119563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200006d760 con 0x7f2000159360 2023-07-06T01:20:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.400+0000 7f20058c8700 1 -- 172.21.15.22:0/929119563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200006d000 con 0x7f2000159360 2023-07-06T01:20:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.401+0000 7f1fe7fff700 1 -- 172.21.15.22:0/929119563 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1ff8007470 con 0x7f2000159360 2023-07-06T01:20:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.401+0000 7f1fe7fff700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1fec06ace0 0x7f1fec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.401+0000 7f1fe7fff700 1 -- 172.21.15.22:0/929119563 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1ff80892e0 con 0x7f2000159360 2023-07-06T01:20:08.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.401+0000 7f1ffeffd700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1fec06ace0 0x7f1fec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.402+0000 7f1ffeffd700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1fec06ace0 0x7f1fec06d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40090b0 tx=0x7f1ff401c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:08.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.406+0000 7f1fe7fff700 1 -- 172.21.15.22:0/929119563 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f200006d000 con 0x7f2000159360 2023-07-06T01:20:08.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.648+0000 7f20058c8700 1 -- 172.21.15.22:0/929119563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f200006d000 con 0x7f2000159360 2023-07-06T01:20:08.652 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.649+0000 7f1fe7fff700 1 -- 172.21.15.22:0/929119563 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4671 (secure 0 0 0) 0x7f200006d000 con 0x7f2000159360 2023-07-06T01:20:08.652 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:08.652 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/2496681609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2496681609},{"type":"v1","addr":"172.21.15.22:6833","nonce":2496681609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2891594418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2891594418},{"type":"v1","addr":"172.21.15.22:6835","nonce":2891594418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/4097705589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4097705589},{"type":"v1","addr":"172.21.15.103:6836","nonce":4097705589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T01:19:25.042193+0000","modified":"2023-07-06T01:19:28.649724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6837/4015300665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":4015300665},{"type":"v1","addr":"172.21.15.103:6837","nonce":4015300665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T01:20:08.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.652+0000 7f20058c8700 1 -- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1fec06ace0 msgr2=0x7f1fec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:08.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.652+0000 7f20058c8700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1fec06ace0 0x7f1fec06d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40090b0 tx=0x7f1ff401c040 comp rx=0 tx=0).stop 2023-07-06T01:20:08.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.652+0000 7f20058c8700 1 -- 172.21.15.22:0/929119563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2000159360 msgr2=0x7f200015a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:08.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.652+0000 7f20058c8700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2000159360 0x7f200015a1f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800cca0 tx=0x7f1ff800a060 comp rx=0 tx=0).stop 2023-07-06T01:20:08.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.652+0000 7f20058c8700 1 -- 172.21.15.22:0/929119563 shutdown_connections 2023-07-06T01:20:08.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.652+0000 7f20058c8700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2000159360 0x7f200015a1f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.652+0000 7f20058c8700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1fec06ace0 0x7f1fec06d190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.652+0000 7f20058c8700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2000070a20 0x7f200015bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.652+0000 7f20058c8700 1 --2- 172.21.15.22:0/929119563 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f20000cc660 0x7f20000b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.652+0000 7f20058c8700 1 -- 172.21.15.22:0/929119563 >> 172.21.15.22:0/929119563 conn(0x7f20000b1890 msgr2=0x7f2000070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:08.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.652+0000 7f20058c8700 1 -- 172.21.15.22:0/929119563 shutdown_connections 2023-07-06T01:20:08.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.652+0000 7f20058c8700 1 -- 172.21.15.22:0/929119563 wait complete. 2023-07-06T01:20:08.656 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2023-07-06T01:20:08.666 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:20:08.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.789+0000 7f30f10b5700 1 Processor -- start 2023-07-06T01:20:08.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.789+0000 7f30f10b5700 1 -- start start 2023-07-06T01:20:08.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.789+0000 7f30f10b5700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30ec06a710 0x7f30ec06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:08.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.789+0000 7f30f10b5700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ec0cbc10 0x7f30ec0cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:08.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.789+0000 7f30f10b5700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ec0af670 con 0x7f30ec0af2a0 2023-07-06T01:20:08.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.789+0000 7f30f10b5700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ec0cd560 con 0x7f30ec06a710 2023-07-06T01:20:08.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.789+0000 7f30f10b5700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec15cab0 con 0x7f30ec0cbc10 2023-07-06T01:20:08.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.789+0000 7f30ea59c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30ec06a710 0x7f30ec06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:08.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.789+0000 7f30ea59c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f30ec06a710 0x7f30ec06caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46828/0 (socket says 172.21.15.22:46828) 2023-07-06T01:20:08.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.789+0000 7f30ea59c700 1 -- 172.21.15.22:0/1522033963 learned_addr learned my addr 172.21.15.22:0/1522033963 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:08.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.790+0000 7f30eb59e700 1 --2- 172.21.15.22:0/1522033963 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ec0cbc10 0x7f30ec0cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:08.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.790+0000 7f30ea59c700 1 -- 172.21.15.22:0/1522033963 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ec0cbc10 msgr2=0x7f30ec0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:08.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.790+0000 7f30ea59c700 1 --2- 172.21.15.22:0/1522033963 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ec0cbc10 0x7f30ec0cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.790+0000 7f30ea59c700 1 -- 172.21.15.22:0/1522033963 >> v1:172.21.15.103:6789/0 conn(0x7f30ec0af2a0 legacy=0x7f30ec06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.790+0000 7f30ea59c700 1 -- 172.21.15.22:0/1522033963 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec06d030 con 0x7f30ec06a710 2023-07-06T01:20:08.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.790+0000 7f30ea59c700 1 --2- 172.21.15.22:0/1522033963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ec06a710 0x7f30ec06caf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f30d40098f0 tx=0x7f30d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ca5e6166cea6d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:08.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.790+0000 7f30e9d9b700 1 -- 172.21.15.22:0/1522033963 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d400b8a0 con 0x7f30ec06a710 2023-07-06T01:20:08.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.790+0000 7f30e9d9b700 1 -- 172.21.15.22:0/1522033963 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d400ba00 con 0x7f30ec06a710 2023-07-06T01:20:08.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.790+0000 7f30e9d9b700 1 -- 172.21.15.22:0/1522033963 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d400bd90 con 0x7f30ec06a710 2023-07-06T01:20:08.799 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.790+0000 7f30f10b5700 1 -- 172.21.15.22:0/1522033963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ec06a710 msgr2=0x7f30ec06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:08.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.790+0000 7f30f10b5700 1 --2- 172.21.15.22:0/1522033963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ec06a710 0x7f30ec06caf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f30d40098f0 tx=0x7f30d4009c00 comp rx=0 tx=0).stop 2023-07-06T01:20:08.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.791+0000 7f30f10b5700 1 -- 172.21.15.22:0/1522033963 shutdown_connections 2023-07-06T01:20:08.800 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.791+0000 7f30f10b5700 1 --2- 172.21.15.22:0/1522033963 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ec0cbc10 0x7f30ec0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.791+0000 7f30f10b5700 1 --2- 172.21.15.22:0/1522033963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ec06a710 0x7f30ec06caf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.791+0000 7f30f10b5700 1 -- 172.21.15.22:0/1522033963 >> 172.21.15.22:0/1522033963 conn(0x7f30ec0b1890 msgr2=0x7f30ec0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:08.801 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.791+0000 7f30f10b5700 1 -- 172.21.15.22:0/1522033963 shutdown_connections 2023-07-06T01:20:08.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.791+0000 7f30f10b5700 1 -- 172.21.15.22:0/1522033963 wait complete. 2023-07-06T01:20:08.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.792+0000 7f30f10b5700 1 Processor -- start 2023-07-06T01:20:08.802 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.792+0000 7f30f10b5700 1 -- start start 2023-07-06T01:20:08.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30f10b5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ec06a710 0x7f30ec0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:08.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30f10b5700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ec0af2a0 0x7f30ec0adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:08.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30f10b5700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ec0cbc10 0x7f30ec15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:08.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30f10b5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec0b4890 con 0x7f30ec06a710 2023-07-06T01:20:08.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30f10b5700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec0b3c80 con 0x7f30ec0af2a0 2023-07-06T01:20:08.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30f10b5700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec0b3f60 con 0x7f30ec0cbc10 2023-07-06T01:20:08.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30ead9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ec06a710 0x7f30ec0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:08.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30ead9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ec06a710 0x7f30ec0b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46832/0 (socket says 172.21.15.22:46832) 2023-07-06T01:20:08.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30ead9d700 1 -- 172.21.15.22:0/1439497917 learned_addr learned my addr 172.21.15.22:0/1439497917 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:08.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30ea59c700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ec0af2a0 0x7f30ec0adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:08.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30eb59e700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ec0cbc10 0x7f30ec15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:08.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30ead9d700 1 -- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ec0cbc10 msgr2=0x7f30ec15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:08.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30ead9d700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ec0cbc10 0x7f30ec15ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30ead9d700 1 -- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ec0af2a0 msgr2=0x7f30ec0adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:08.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30ead9d700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ec0af2a0 0x7f30ec0adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:08.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30ead9d700 1 -- 172.21.15.22:0/1439497917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d4009550 con 0x7f30ec06a710 2023-07-06T01:20:08.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30eb59e700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ec0cbc10 0x7f30ec15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:08.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.793+0000 7f30ead9d700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ec06a710 0x7f30ec0b5530 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c4e0 tx=0x7f30dc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:08.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.794+0000 7f30e37fe700 1 -- 172.21.15.22:0/1439497917 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0052d0 con 0x7f30ec06a710 2023-07-06T01:20:08.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.794+0000 7f30e37fe700 1 -- 172.21.15.22:0/1439497917 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc005430 con 0x7f30ec06a710 2023-07-06T01:20:08.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.794+0000 7f30f10b5700 1 -- 172.21.15.22:0/1439497917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ec0b4240 con 0x7f30ec06a710 2023-07-06T01:20:08.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.794+0000 7f30e37fe700 1 -- 172.21.15.22:0/1439497917 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0038c0 con 0x7f30ec06a710 2023-07-06T01:20:08.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.794+0000 7f30f10b5700 1 -- 172.21.15.22:0/1439497917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ec0b5a70 con 0x7f30ec06a710 2023-07-06T01:20:08.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.795+0000 7f30f10b5700 1 -- 172.21.15.22:0/1439497917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ec0744b0 con 0x7f30ec06a710 2023-07-06T01:20:08.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.795+0000 7f30e37fe700 1 -- 172.21.15.22:0/1439497917 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f30dc003a20 con 0x7f30ec06a710 2023-07-06T01:20:08.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.795+0000 7f30e37fe700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f30cc06ace0 0x7f30cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:08.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.797+0000 7f30e37fe700 1 -- 172.21.15.22:0/1439497917 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f30dc0889b0 con 0x7f30ec06a710 2023-07-06T01:20:08.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.797+0000 7f30ea59c700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f30cc06ace0 0x7f30cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:08.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.797+0000 7f30ea59c700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f30cc06ace0 0x7f30cc06d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f30d40098f0 tx=0x7f30d4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:08.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.801+0000 7f30e37fe700 1 -- 172.21.15.22:0/1439497917 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f30dc053a50 con 0x7f30ec06a710 2023-07-06T01:20:09.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:08.999+0000 7f30f10b5700 1 -- 172.21.15.22:0/1439497917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30ec0712e0 con 0x7f30ec06a710 2023-07-06T01:20:09.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.000+0000 7f30e37fe700 1 -- 172.21.15.22:0/1439497917 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7f30dc057070 con 0x7f30ec06a710 2023-07-06T01:20:09.002 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:09.002 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:20:09.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.002+0000 7f30f10b5700 1 -- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f30cc06ace0 msgr2=0x7f30cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.002+0000 7f30f10b5700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f30cc06ace0 0x7f30cc06d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f30d40098f0 tx=0x7f30d4008030 comp rx=0 tx=0).stop 2023-07-06T01:20:09.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.002+0000 7f30f10b5700 1 -- 172.21.15.22:0/1439497917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ec06a710 msgr2=0x7f30ec0b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.002+0000 7f30f10b5700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ec06a710 0x7f30ec0b5530 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c4e0 tx=0x7f30dc00c8a0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.002+0000 7f30f10b5700 1 -- 172.21.15.22:0/1439497917 shutdown_connections 2023-07-06T01:20:09.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.002+0000 7f30f10b5700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30ec0cbc10 0x7f30ec15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.002+0000 7f30f10b5700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30ec06a710 0x7f30ec0b5530 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.002+0000 7f30f10b5700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f30cc06ace0 0x7f30cc06d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.002+0000 7f30f10b5700 1 --2- 172.21.15.22:0/1439497917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30ec0af2a0 0x7f30ec0adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.002+0000 7f30f10b5700 1 -- 172.21.15.22:0/1439497917 >> 172.21.15.22:0/1439497917 conn(0x7f30ec0b1890 msgr2=0x7f30ec0cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.002+0000 7f30f10b5700 1 -- 172.21.15.22:0/1439497917 shutdown_connections 2023-07-06T01:20:09.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.002+0000 7f30f10b5700 1 -- 172.21.15.22:0/1439497917 wait complete. 2023-07-06T01:20:09.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T01:19:22.873255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T01:19:24.019088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:20:09.017 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:20:09.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.143+0000 7f7536a3d700 1 Processor -- start 2023-07-06T01:20:09.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.143+0000 7f7536a3d700 1 -- start start 2023-07-06T01:20:09.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.144+0000 7f7536a3d700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f753015a710 0x7f75300ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.144+0000 7f7536a3d700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75300af270 con 0x7f75300cef70 2023-07-06T01:20:09.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.144+0000 7f7536a3d700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75300af550 con 0x7f75300aecb0 2023-07-06T01:20:09.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.144+0000 7f7536a3d700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75300af830 con 0x7f753015a710 2023-07-06T01:20:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.144+0000 7f7534fda700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f75300cef70 0x7f75300cbae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37588/0 (socket says 172.21.15.22:37588) 2023-07-06T01:20:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.144+0000 7f7534fda700 1 -- 172.21.15.22:0/3171155563 learned_addr learned my addr 172.21.15.22:0/3171155563 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752ffff700 1 --2- 172.21.15.22:0/3171155563 >> v2:172.21.15.103:3300/0 conn(0x7f753015a710 0x7f75300ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224303522 0 0) 0x7f75300af270 con 0x7f75300cef70 2023-07-06T01:20:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f751c0035b0 con 0x7f75300cef70 2023-07-06T01:20:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3934309827 0 0) 0x7f751c0035b0 con 0x7f75300cef70 2023-07-06T01:20:09.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 >> v2:172.21.15.103:3300/0 conn(0x7f753015a710 msgr2=0x7f75300ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 --2- 172.21.15.22:0/3171155563 >> v2:172.21.15.103:3300/0 conn(0x7f753015a710 0x7f75300ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 >> v1:172.21.15.103:6789/0 conn(0x7f75300aecb0 legacy=0x7f75300ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75300cb2e0 con 0x7f75300cef70 2023-07-06T01:20:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f75280019f0 con 0x7f75300cef70 2023-07-06T01:20:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752ffff700 1 --2- 172.21.15.22:0/3171155563 >> v2:172.21.15.103:3300/0 conn(0x7f753015a710 0x7f75300ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 >> v1:172.21.15.22:6789/0 conn(0x7f75300cef70 legacy=0x7f75300cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 --2- 172.21.15.22:0/3171155563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c003a00 0x7f751c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 --2- 172.21.15.22:0/3171155563 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f751c004850 0x7f751c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 --2- 172.21.15.22:0/3171155563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f751c005200 0x7f751c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75300af830 con 0x7f751c005200 2023-07-06T01:20:09.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c00c080 con 0x7f751c003a00 2023-07-06T01:20:09.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751c00c240 con 0x7f751c004850 2023-07-06T01:20:09.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f7534fda700 1 --2- 172.21.15.22:0/3171155563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f751c005200 0x7f751c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752ffff700 1 --2- 172.21.15.22:0/3171155563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c003a00 0x7f751c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.145+0000 7f752f7fe700 1 --2- 172.21.15.22:0/3171155563 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f751c004850 0x7f751c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f7534fda700 1 -- 172.21.15.22:0/3171155563 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f751c004850 msgr2=0x7f751c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f7534fda700 1 --2- 172.21.15.22:0/3171155563 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f751c004850 0x7f751c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f7534fda700 1 -- 172.21.15.22:0/3171155563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c003a00 msgr2=0x7f751c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f7534fda700 1 --2- 172.21.15.22:0/3171155563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f751c003a00 0x7f751c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f7534fda700 1 -- 172.21.15.22:0/3171155563 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c0035b0 con 0x7f751c005200 2023-07-06T01:20:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f7534fda700 1 -- 172.21.15.22:0/3171155563 reap_dead start 2023-07-06T01:20:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f7534fda700 1 --2- 172.21.15.22:0/3171155563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f751c005200 0x7f751c00ba10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f752800a800 tx=0x7f751c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7671e75b865274b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:09.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752800a2f0 con 0x7f751c005200 2023-07-06T01:20:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7528012620 con 0x7f751c005200 2023-07-06T01:20:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f752effd700 1 -- 172.21.15.22:0/3171155563 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75280129d0 con 0x7f751c005200 2023-07-06T01:20:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f7536a3d700 1 -- 172.21.15.22:0/3171155563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f751c005200 msgr2=0x7f751c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f7536a3d700 1 --2- 172.21.15.22:0/3171155563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f751c005200 0x7f751c00ba10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f752800a800 tx=0x7f751c005880 comp rx=0 tx=0).stop 2023-07-06T01:20:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f7536a3d700 1 -- 172.21.15.22:0/3171155563 shutdown_connections 2023-07-06T01:20:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f7536a3d700 1 --2- 172.21.15.22:0/3171155563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f751c005200 0x7f751c00ba10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.146+0000 7f7536a3d700 1 -- 172.21.15.22:0/3171155563 >> 172.21.15.22:0/3171155563 conn(0x7f75300999f0 msgr2=0x7f7530099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:09.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.147+0000 7f7536a3d700 1 -- 172.21.15.22:0/3171155563 shutdown_connections 2023-07-06T01:20:09.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.147+0000 7f7536a3d700 1 -- 172.21.15.22:0/3171155563 wait complete. 2023-07-06T01:20:09.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.148+0000 7f7536a3d700 1 Processor -- start 2023-07-06T01:20:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.148+0000 7f7536a3d700 1 -- start start 2023-07-06T01:20:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f7536a3d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f751c005200 0x7f75301ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f7536a3d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75301f0700 0x7f75300c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f7536a3d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75300c6490 0x7f7530151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f7536a3d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75300c75f0 con 0x7f751c005200 2023-07-06T01:20:09.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f7536a3d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75301f10c0 con 0x7f75301f0700 2023-07-06T01:20:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f7536a3d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75301f13a0 con 0x7f75300c6490 2023-07-06T01:20:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f752ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f751c005200 0x7f75301ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f752ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f751c005200 0x7f75301ed9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46852/0 (socket says 172.21.15.22:46852) 2023-07-06T01:20:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f752ffff700 1 -- 172.21.15.22:0/1089072117 learned_addr learned my addr 172.21.15.22:0/1089072117 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f752f7fe700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75301f0700 0x7f75300c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f7534fda700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75300c6490 0x7f7530151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f752ffff700 1 -- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75300c6490 msgr2=0x7f7530151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f752ffff700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75300c6490 0x7f7530151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f752ffff700 1 -- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75301f0700 msgr2=0x7f75300c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f752ffff700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75301f0700 0x7f75300c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f752ffff700 1 -- 172.21.15.22:0/1089072117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75301f1680 con 0x7f751c005200 2023-07-06T01:20:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f752ffff700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f751c005200 0x7f75301ed9b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7520010e80 tx=0x7f752000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.149+0000 7f752cff9700 1 -- 172.21.15.22:0/1089072117 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520017040 con 0x7f751c005200 2023-07-06T01:20:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.150+0000 7f752cff9700 1 -- 172.21.15.22:0/1089072117 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752000bd90 con 0x7f751c005200 2023-07-06T01:20:09.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.150+0000 7f7536a3d700 1 -- 172.21.15.22:0/1089072117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75301f1960 con 0x7f751c005200 2023-07-06T01:20:09.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.150+0000 7f752cff9700 1 -- 172.21.15.22:0/1089072117 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520004240 con 0x7f751c005200 2023-07-06T01:20:09.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.150+0000 7f7536a3d700 1 -- 172.21.15.22:0/1089072117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75301f0b70 con 0x7f751c005200 2023-07-06T01:20:09.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.151+0000 7f7536a3d700 1 -- 172.21.15.22:0/1089072117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75300b3b50 con 0x7f751c005200 2023-07-06T01:20:09.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.151+0000 7f752cff9700 1 -- 172.21.15.22:0/1089072117 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f75200043a0 con 0x7f751c005200 2023-07-06T01:20:09.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.151+0000 7f752cff9700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f751006ace0 0x7f751006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.151+0000 7f752cff9700 1 -- 172.21.15.22:0/1089072117 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f752008ceb0 con 0x7f751c005200 2023-07-06T01:20:09.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.152+0000 7f752f7fe700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f751006ace0 0x7f751006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.152+0000 7f752f7fe700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f751006ace0 0x7f751006d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f75300b33f0 tx=0x7f7518009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:09.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.156+0000 7f752cff9700 1 -- 172.21.15.22:0/1089072117 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7520057f60 con 0x7f751c005200 2023-07-06T01:20:09.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.355+0000 7f7536a3d700 1 -- 172.21.15.22:0/1089072117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f75301f0d30 con 0x7f751c005200 2023-07-06T01:20:09.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.355+0000 7f752cff9700 1 -- 172.21.15.22:0/1089072117 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f75301f0d30 con 0x7f751c005200 2023-07-06T01:20:09.358 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:20:09.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.357+0000 7f7536a3d700 1 -- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f751006ace0 msgr2=0x7f751006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.357+0000 7f7536a3d700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f751006ace0 0x7f751006d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f75300b33f0 tx=0x7f7518009040 comp rx=0 tx=0).stop 2023-07-06T01:20:09.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.357+0000 7f7536a3d700 1 -- 172.21.15.22:0/1089072117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f751c005200 msgr2=0x7f75301ed9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.357+0000 7f7536a3d700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f751c005200 0x7f75301ed9b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7520010e80 tx=0x7f752000b420 comp rx=0 tx=0).stop 2023-07-06T01:20:09.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.358+0000 7f7536a3d700 1 -- 172.21.15.22:0/1089072117 shutdown_connections 2023-07-06T01:20:09.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.358+0000 7f7536a3d700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75300c6490 0x7f7530151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.358+0000 7f7536a3d700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f751c005200 0x7f75301ed9b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.358+0000 7f7536a3d700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f751006ace0 0x7f751006d190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.358+0000 7f7536a3d700 1 --2- 172.21.15.22:0/1089072117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75301f0700 0x7f75300c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.358+0000 7f7536a3d700 1 -- 172.21.15.22:0/1089072117 >> 172.21.15.22:0/1089072117 conn(0x7f75300999f0 msgr2=0x7f75300adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.358+0000 7f7536a3d700 1 -- 172.21.15.22:0/1089072117 shutdown_connections 2023-07-06T01:20:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.358+0000 7f7536a3d700 1 -- 172.21.15.22:0/1089072117 wait complete. 2023-07-06T01:20:09.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:20:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.497+0000 7fc66ce42700 1 Processor -- start 2023-07-06T01:20:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.497+0000 7fc66ce42700 1 -- start start 2023-07-06T01:20:09.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.498+0000 7fc66ce42700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc66815d9e0 0x7fc66806d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.498+0000 7fc66ce42700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6680cf230 con 0x7fc6680ccd70 2023-07-06T01:20:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.498+0000 7fc66ce42700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6680ce620 con 0x7fc668159360 2023-07-06T01:20:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.498+0000 7fc66ce42700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6680ce900 con 0x7fc66815d9e0 2023-07-06T01:20:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.498+0000 7fc666d9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc66815d9e0 0x7fc66806d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.498+0000 7fc666d9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc66815d9e0 0x7fc66806d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46864/0 (socket says 172.21.15.22:46864) 2023-07-06T01:20:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.498+0000 7fc665d9b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc6680ccd70 0x7fc66806d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37606/0 (socket says 172.21.15.22:37606) 2023-07-06T01:20:09.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.498+0000 7fc666d9d700 1 -- 172.21.15.22:0/1575228210 learned_addr learned my addr 172.21.15.22:0/1575228210 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.498+0000 7fc666d9d700 1 -- 172.21.15.22:0/1575228210 >> v1:172.21.15.103:6789/0 conn(0x7fc668159360 legacy=0x7fc6680cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.498+0000 7fc666d9d700 1 -- 172.21.15.22:0/1575228210 >> v1:172.21.15.22:6789/0 conn(0x7fc6680ccd70 legacy=0x7fc66806d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.498+0000 7fc666d9d700 1 -- 172.21.15.22:0/1575228210 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680cebe0 con 0x7fc66815d9e0 2023-07-06T01:20:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.498+0000 7fc66559a700 1 -- 172.21.15.22:0/1575228210 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299890188 0 0) 0x7fc6680cf230 con 0x7fc6680ccd70 2023-07-06T01:20:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.498+0000 7fc666d9d700 1 --2- 172.21.15.22:0/1575228210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc66815d9e0 0x7fc66806d750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc660009e50 tx=0x7fc66000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85722e2b80a3ef80 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:09.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.499+0000 7fc66559a700 1 -- 172.21.15.22:0/1575228210 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66000e070 con 0x7fc66815d9e0 2023-07-06T01:20:09.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.499+0000 7fc66559a700 1 -- 172.21.15.22:0/1575228210 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66000b9c0 con 0x7fc66815d9e0 2023-07-06T01:20:09.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.499+0000 7fc66559a700 1 -- 172.21.15.22:0/1575228210 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66000bd70 con 0x7fc66815d9e0 2023-07-06T01:20:09.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.499+0000 7fc66ce42700 1 -- 172.21.15.22:0/1575228210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc66815d9e0 msgr2=0x7fc66806d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.499+0000 7fc66ce42700 1 --2- 172.21.15.22:0/1575228210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc66815d9e0 0x7fc66806d750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc660009e50 tx=0x7fc66000b330 comp rx=0 tx=0).stop 2023-07-06T01:20:09.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.499+0000 7fc66ce42700 1 -- 172.21.15.22:0/1575228210 shutdown_connections 2023-07-06T01:20:09.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.499+0000 7fc66ce42700 1 --2- 172.21.15.22:0/1575228210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc66815d9e0 0x7fc66806d750 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.499+0000 7fc66ce42700 1 -- 172.21.15.22:0/1575228210 >> 172.21.15.22:0/1575228210 conn(0x7fc6680b1890 msgr2=0x7fc6680b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:09.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.499+0000 7fc66ce42700 1 -- 172.21.15.22:0/1575228210 shutdown_connections 2023-07-06T01:20:09.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.499+0000 7fc66ce42700 1 -- 172.21.15.22:0/1575228210 wait complete. 2023-07-06T01:20:09.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.501+0000 7fc66ce42700 1 Processor -- start 2023-07-06T01:20:09.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.501+0000 7fc66ce42700 1 -- start start 2023-07-06T01:20:09.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.501+0000 7fc66ce42700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6680ccd70 0x7fc6680b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.501+0000 7fc66ce42700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc668159360 0x7fc66806a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.501+0000 7fc66ce42700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc66815d9e0 0x7fc6680aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.501+0000 7fc66ce42700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66806a4a0 con 0x7fc6680ccd70 2023-07-06T01:20:09.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.501+0000 7fc66ce42700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680b59b0 con 0x7fc66815d9e0 2023-07-06T01:20:09.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.501+0000 7fc66ce42700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680b53f0 con 0x7fc668159360 2023-07-06T01:20:09.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.501+0000 7fc66659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6680ccd70 0x7fc6680b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc66659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6680ccd70 0x7fc6680b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46866/0 (socket says 172.21.15.22:46866) 2023-07-06T01:20:09.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc66659c700 1 -- 172.21.15.22:0/3466303887 learned_addr learned my addr 172.21.15.22:0/3466303887 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:09.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc66659c700 1 -- 172.21.15.22:0/3466303887 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc668159360 msgr2=0x7fc66806a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc66659c700 1 --2- 172.21.15.22:0/3466303887 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc668159360 0x7fc66806a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc66659c700 1 -- 172.21.15.22:0/3466303887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc66815d9e0 msgr2=0x7fc6680aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc666d9d700 1 --2- 172.21.15.22:0/3466303887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc66815d9e0 0x7fc6680aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc66659c700 1 --2- 172.21.15.22:0/3466303887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc66815d9e0 0x7fc6680aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc66659c700 1 -- 172.21.15.22:0/3466303887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc660009b30 con 0x7fc6680ccd70 2023-07-06T01:20:09.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc66659c700 1 --2- 172.21.15.22:0/3466303887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6680ccd70 0x7fc6680b4ba0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc65800c4e0 tx=0x7fc65800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:09.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc656ffd700 1 -- 172.21.15.22:0/3466303887 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6580052d0 con 0x7fc6680ccd70 2023-07-06T01:20:09.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc656ffd700 1 -- 172.21.15.22:0/3466303887 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc658005430 con 0x7fc6680ccd70 2023-07-06T01:20:09.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc66ce42700 1 -- 172.21.15.22:0/3466303887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6680b56d0 con 0x7fc6680ccd70 2023-07-06T01:20:09.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc656ffd700 1 -- 172.21.15.22:0/3466303887 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6580038c0 con 0x7fc6680ccd70 2023-07-06T01:20:09.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.502+0000 7fc66ce42700 1 -- 172.21.15.22:0/3466303887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6680b50e0 con 0x7fc6680ccd70 2023-07-06T01:20:09.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.503+0000 7fc66ce42700 1 -- 172.21.15.22:0/3466303887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66806a780 con 0x7fc6680ccd70 2023-07-06T01:20:09.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.504+0000 7fc656ffd700 1 -- 172.21.15.22:0/3466303887 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc658003a20 con 0x7fc6680ccd70 2023-07-06T01:20:09.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.504+0000 7fc656ffd700 1 --2- 172.21.15.22:0/3466303887 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc64806ace0 0x7fc64806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.504+0000 7fc656ffd700 1 -- 172.21.15.22:0/3466303887 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc658088960 con 0x7fc6680ccd70 2023-07-06T01:20:09.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.504+0000 7fc665d9b700 1 --2- 172.21.15.22:0/3466303887 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc64806ace0 0x7fc64806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.505+0000 7fc665d9b700 1 --2- 172.21.15.22:0/3466303887 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc64806ace0 0x7fc64806d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc65c000c00 tx=0x7fc65c007990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:09.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.508+0000 7fc656ffd700 1 -- 172.21.15.22:0/3466303887 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc658053a10 con 0x7fc6680ccd70 2023-07-06T01:20:09.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.703+0000 7fc66ce42700 1 -- 172.21.15.22:0/3466303887 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6680744b0 con 0x7fc6680ccd70 2023-07-06T01:20:09.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.704+0000 7fc656ffd700 1 -- 172.21.15.22:0/3466303887 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc658017020 con 0x7fc6680ccd70 2023-07-06T01:20:09.706 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:20:09.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.706+0000 7fc66ce42700 1 -- 172.21.15.22:0/3466303887 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc64806ace0 msgr2=0x7fc64806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.706+0000 7fc66ce42700 1 --2- 172.21.15.22:0/3466303887 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc64806ace0 0x7fc64806d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc65c000c00 tx=0x7fc65c007990 comp rx=0 tx=0).stop 2023-07-06T01:20:09.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.706+0000 7fc66ce42700 1 -- 172.21.15.22:0/3466303887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6680ccd70 msgr2=0x7fc6680b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.706+0000 7fc66ce42700 1 --2- 172.21.15.22:0/3466303887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6680ccd70 0x7fc6680b4ba0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc65800c4e0 tx=0x7fc65800c8a0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.706+0000 7fc66ce42700 1 -- 172.21.15.22:0/3466303887 shutdown_connections 2023-07-06T01:20:09.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.706+0000 7fc66ce42700 1 --2- 172.21.15.22:0/3466303887 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fc64806ace0 0x7fc64806d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.706+0000 7fc66ce42700 1 --2- 172.21.15.22:0/3466303887 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc66815d9e0 0x7fc6680aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.706+0000 7fc66ce42700 1 --2- 172.21.15.22:0/3466303887 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6680ccd70 0x7fc6680b4ba0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.706+0000 7fc66ce42700 1 --2- 172.21.15.22:0/3466303887 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc668159360 0x7fc66806a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.706+0000 7fc66ce42700 1 -- 172.21.15.22:0/3466303887 >> 172.21.15.22:0/3466303887 conn(0x7fc6680b1890 msgr2=0x7fc6680b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:09.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.706+0000 7fc66ce42700 1 -- 172.21.15.22:0/3466303887 shutdown_connections 2023-07-06T01:20:09.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.706+0000 7fc66ce42700 1 -- 172.21.15.22:0/3466303887 wait complete. 2023-07-06T01:20:09.719 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T01:20:09.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.845+0000 7f1e1d1ac700 1 Processor -- start 2023-07-06T01:20:09.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.845+0000 7f1e1d1ac700 1 -- start start 2023-07-06T01:20:09.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.846+0000 7f1e1d1ac700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1e180af2a0 0x7f1e1806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.846+0000 7f1e1d1ac700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1e1806a000 0x7f1e1806a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.846+0000 7f1e1d1ac700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e180af670 con 0x7f1e180cbc10 2023-07-06T01:20:09.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.846+0000 7f1e1d1ac700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1806a9e0 con 0x7f1e180af2a0 2023-07-06T01:20:09.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.846+0000 7f1e1d1ac700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180cc7a0 con 0x7f1e1806a000 2023-07-06T01:20:09.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.846+0000 7f1e16d9d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1e180af2a0 0x7f1e1806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.846+0000 7f1e1759e700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1e180cbc10 0x7f1e180cc090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37622/0 (socket says 172.21.15.22:37622) 2023-07-06T01:20:09.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.846+0000 7f1e1759e700 1 -- 172.21.15.22:0/2082949438 learned_addr learned my addr 172.21.15.22:0/2082949438 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:09.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.847+0000 7f1e1659c700 1 --2- 172.21.15.22:0/2082949438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1e1806a000 0x7f1e1806a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.847+0000 7f1e16d9d700 1 -- 172.21.15.22:0/2082949438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1e1806a000 msgr2=0x7f1e1806a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:09.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.847+0000 7f1e16d9d700 1 --2- 172.21.15.22:0/2082949438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1e1806a000 0x7f1e1806a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.847+0000 7f1e16d9d700 1 -- 172.21.15.22:0/2082949438 >> v1:172.21.15.22:6789/0 conn(0x7f1e180cbc10 legacy=0x7f1e180cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.847+0000 7f1e16d9d700 1 -- 172.21.15.22:0/2082949438 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1806d030 con 0x7f1e180af2a0 2023-07-06T01:20:09.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.847+0000 7f1e15d9b700 1 -- 172.21.15.22:0/2082949438 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688171018 0 0) 0x7f1e180af670 con 0x7f1e180cbc10 2023-07-06T01:20:09.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.847+0000 7f1e16d9d700 1 --2- 172.21.15.22:0/2082949438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e180af2a0 0x7f1e1806caf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009840 tx=0x7f1e0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41eaac1ea3eaa687 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:09.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.847+0000 7f1e15d9b700 1 -- 172.21.15.22:0/2082949438 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c011830 con 0x7f1e180af2a0 2023-07-06T01:20:09.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.847+0000 7f1e15d9b700 1 -- 172.21.15.22:0/2082949438 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c011990 con 0x7f1e180af2a0 2023-07-06T01:20:09.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.847+0000 7f1e15d9b700 1 -- 172.21.15.22:0/2082949438 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c011d20 con 0x7f1e180af2a0 2023-07-06T01:20:09.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.848+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/2082949438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e180af2a0 msgr2=0x7f1e1806caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.848+0000 7f1e1d1ac700 1 --2- 172.21.15.22:0/2082949438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e180af2a0 0x7f1e1806caf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009840 tx=0x7f1e0c009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:09.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.848+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/2082949438 shutdown_connections 2023-07-06T01:20:09.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.848+0000 7f1e1d1ac700 1 --2- 172.21.15.22:0/2082949438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e180af2a0 0x7f1e1806caf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.848+0000 7f1e1d1ac700 1 --2- 172.21.15.22:0/2082949438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1e1806a000 0x7f1e1806a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.848+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/2082949438 >> 172.21.15.22:0/2082949438 conn(0x7f1e180b1890 msgr2=0x7f1e180b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:09.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.848+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/2082949438 shutdown_connections 2023-07-06T01:20:09.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.848+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/2082949438 wait complete. 2023-07-06T01:20:09.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.849+0000 7f1e1d1ac700 1 Processor -- start 2023-07-06T01:20:09.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.849+0000 7f1e1d1ac700 1 -- start start 2023-07-06T01:20:09.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1d1ac700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1e1806a000 0x7f1e18159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1d1ac700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e180af2a0 0x7f1e1815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1d1ac700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e180cbc10 0x7f1e1815aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1d1ac700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180b30e0 con 0x7f1e180cbc10 2023-07-06T01:20:09.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1d1ac700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1815a360 con 0x7f1e180af2a0 2023-07-06T01:20:09.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1d1ac700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1815ce40 con 0x7f1e1806a000 2023-07-06T01:20:09.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1759e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e180cbc10 0x7f1e1815aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1759e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e180cbc10 0x7f1e1815aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46884/0 (socket says 172.21.15.22:46884) 2023-07-06T01:20:09.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1759e700 1 -- 172.21.15.22:0/3516459124 learned_addr learned my addr 172.21.15.22:0/3516459124 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:09.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e16d9d700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1e1806a000 0x7f1e18159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1659c700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e180af2a0 0x7f1e1815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1759e700 1 -- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1e1806a000 msgr2=0x7f1e18159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1759e700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1e1806a000 0x7f1e18159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1759e700 1 -- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e180af2a0 msgr2=0x7f1e1815a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:09.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1759e700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e180af2a0 0x7f1e1815a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:09.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1759e700 1 -- 172.21.15.22:0/3516459124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c0094a0 con 0x7f1e180cbc10 2023-07-06T01:20:09.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e16d9d700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1e1806a000 0x7f1e18159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:09.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.850+0000 7f1e1659c700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e180af2a0 0x7f1e1815a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:09.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.851+0000 7f1e1759e700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e180cbc10 0x7f1e1815aaf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000cba0 tx=0x7f1e1000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:09.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.851+0000 7f1dff7fe700 1 -- 172.21.15.22:0/3516459124 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000ab80 con 0x7f1e180cbc10 2023-07-06T01:20:09.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.851+0000 7f1dff7fe700 1 -- 172.21.15.22:0/3516459124 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1000ad00 con 0x7f1e180cbc10 2023-07-06T01:20:09.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.851+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/3516459124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1815d120 con 0x7f1e180cbc10 2023-07-06T01:20:09.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.851+0000 7f1dff7fe700 1 -- 172.21.15.22:0/3516459124 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10007310 con 0x7f1e180cbc10 2023-07-06T01:20:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.851+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/3516459124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1815d6e0 con 0x7f1e180cbc10 2023-07-06T01:20:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.852+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/3516459124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1815d900 con 0x7f1e180cbc10 2023-07-06T01:20:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.852+0000 7f1dff7fe700 1 -- 172.21.15.22:0/3516459124 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1e10007470 con 0x7f1e180cbc10 2023-07-06T01:20:09.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.853+0000 7f1dff7fe700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1e0406ace0 0x7f1e0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:09.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.853+0000 7f1dff7fe700 1 -- 172.21.15.22:0/3516459124 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1e10089250 con 0x7f1e180cbc10 2023-07-06T01:20:09.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.853+0000 7f1e16d9d700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1e0406ace0 0x7f1e0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:09.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.854+0000 7f1e16d9d700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1e0406ace0 0x7f1e0406d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c000c00 tx=0x7f1e0c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:09.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:09.857+0000 7f1dff7fe700 1 -- 172.21.15.22:0/3516459124 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1e10054390 con 0x7f1e180cbc10 2023-07-06T01:20:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.053+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/3516459124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e1815ca90 con 0x7f1e180cbc10 2023-07-06T01:20:10.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.054+0000 7f1dff7fe700 1 -- 172.21.15.22:0/3516459124 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e1815ca90 con 0x7f1e180cbc10 2023-07-06T01:20:10.056 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-07-06T01:20:10.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.056+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1e0406ace0 msgr2=0x7f1e0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.056+0000 7f1e1d1ac700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1e0406ace0 0x7f1e0406d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c000c00 tx=0x7f1e0c008030 comp rx=0 tx=0).stop 2023-07-06T01:20:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.056+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/3516459124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e180cbc10 msgr2=0x7f1e1815aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.056+0000 7f1e1d1ac700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e180cbc10 0x7f1e1815aaf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000cba0 tx=0x7f1e1000cf60 comp rx=0 tx=0).stop 2023-07-06T01:20:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.056+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/3516459124 shutdown_connections 2023-07-06T01:20:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.056+0000 7f1e1d1ac700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1e180cbc10 0x7f1e1815aaf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.056+0000 7f1e1d1ac700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1e1806a000 0x7f1e18159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.056+0000 7f1e1d1ac700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1e0406ace0 0x7f1e0406d190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.056+0000 7f1e1d1ac700 1 --2- 172.21.15.22:0/3516459124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e180af2a0 0x7f1e1815a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.056+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/3516459124 >> 172.21.15.22:0/3516459124 conn(0x7f1e180b1890 msgr2=0x7f1e1806d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:10.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.056+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/3516459124 shutdown_connections 2023-07-06T01:20:10.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.056+0000 7f1e1d1ac700 1 -- 172.21.15.22:0/3516459124 wait complete. 2023-07-06T01:20:10.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T01:20:10.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.196+0000 7f97aaf9f700 1 Processor -- start 2023-07-06T01:20:10.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.196+0000 7f97aaf9f700 1 -- start start 2023-07-06T01:20:10.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.197+0000 7f97aaf9f700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f97a406a710 0x7f97a406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.197+0000 7f97aaf9f700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a40af670 con 0x7f97a40af2a0 2023-07-06T01:20:10.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.197+0000 7f97aaf9f700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a406d030 con 0x7f97a40cbc10 2023-07-06T01:20:10.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.197+0000 7f97aaf9f700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a40cc780 con 0x7f97a406a710 2023-07-06T01:20:10.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.197+0000 7f97a3fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f97a406a710 0x7f97a406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.197+0000 7f97a8d3b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f97a40af2a0 0x7f97a406a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37632/0 (socket says 172.21.15.22:37632) 2023-07-06T01:20:10.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.197+0000 7f97a3fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f97a406a710 0x7f97a406caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46890/0 (socket says 172.21.15.22:46890) 2023-07-06T01:20:10.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.197+0000 7f97a8d3b700 1 -- 172.21.15.22:0/3655329533 learned_addr learned my addr 172.21.15.22:0/3655329533 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:10.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.197+0000 7f97a3fff700 1 -- 172.21.15.22:0/3655329533 >> v1:172.21.15.103:6789/0 conn(0x7f97a40cbc10 legacy=0x7f97a40cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.197+0000 7f97a3fff700 1 -- 172.21.15.22:0/3655329533 >> v1:172.21.15.22:6789/0 conn(0x7f97a40af2a0 legacy=0x7f97a406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.197+0000 7f97a3fff700 1 -- 172.21.15.22:0/3655329533 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a40cd560 con 0x7f97a406a710 2023-07-06T01:20:10.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.198+0000 7f97a3fff700 1 --2- 172.21.15.22:0/3655329533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97a406a710 0x7f97a406caf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9790009840 tx=0x7f9790009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2591ef5c89ad786 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:10.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.198+0000 7f97a37fe700 1 -- 172.21.15.22:0/3655329533 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790011830 con 0x7f97a406a710 2023-07-06T01:20:10.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.198+0000 7f97a37fe700 1 -- 172.21.15.22:0/3655329533 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9790011990 con 0x7f97a406a710 2023-07-06T01:20:10.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.198+0000 7f97a37fe700 1 -- 172.21.15.22:0/3655329533 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790011d20 con 0x7f97a406a710 2023-07-06T01:20:10.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.198+0000 7f97aaf9f700 1 -- 172.21.15.22:0/3655329533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97a406a710 msgr2=0x7f97a406caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.198+0000 7f97aaf9f700 1 --2- 172.21.15.22:0/3655329533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97a406a710 0x7f97a406caf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9790009840 tx=0x7f9790009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:10.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.198+0000 7f97aaf9f700 1 -- 172.21.15.22:0/3655329533 shutdown_connections 2023-07-06T01:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.198+0000 7f97aaf9f700 1 --2- 172.21.15.22:0/3655329533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97a406a710 0x7f97a406caf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.198+0000 7f97aaf9f700 1 -- 172.21.15.22:0/3655329533 >> 172.21.15.22:0/3655329533 conn(0x7f97a40b1890 msgr2=0x7f97a40b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.198+0000 7f97aaf9f700 1 -- 172.21.15.22:0/3655329533 shutdown_connections 2023-07-06T01:20:10.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.198+0000 7f97aaf9f700 1 -- 172.21.15.22:0/3655329533 wait complete. 2023-07-06T01:20:10.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.200+0000 7f97aaf9f700 1 Processor -- start 2023-07-06T01:20:10.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.200+0000 7f97aaf9f700 1 -- start start 2023-07-06T01:20:10.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.200+0000 7f97aaf9f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97a406a710 0x7f97a40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.200+0000 7f97aaf9f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97a40af2a0 0x7f97a41f1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.200+0000 7f97aaf9f700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f97a40cbc10 0x7f97a415e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.200+0000 7f97aaf9f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a415de90 con 0x7f97a40af2a0 2023-07-06T01:20:10.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.200+0000 7f97aaf9f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a415d280 con 0x7f97a406a710 2023-07-06T01:20:10.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.200+0000 7f97aaf9f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a415d560 con 0x7f97a40cbc10 2023-07-06T01:20:10.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97a40af2a0 0x7f97a41f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97a40af2a0 0x7f97a41f1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46896/0 (socket says 172.21.15.22:46896) 2023-07-06T01:20:10.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a3fff700 1 -- 172.21.15.22:0/2439575970 learned_addr learned my addr 172.21.15.22:0/2439575970 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:10.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a8d3b700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97a406a710 0x7f97a40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a3fff700 1 -- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f97a40cbc10 msgr2=0x7f97a415e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a953c700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f97a40cbc10 0x7f97a415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a3fff700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f97a40cbc10 0x7f97a415e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a3fff700 1 -- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97a406a710 msgr2=0x7f97a40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a3fff700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97a406a710 0x7f97a40b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a3fff700 1 -- 172.21.15.22:0/2439575970 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97900094a0 con 0x7f97a40af2a0 2023-07-06T01:20:10.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a8d3b700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97a406a710 0x7f97a40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:10.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a3fff700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97a40af2a0 0x7f97a41f1970 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f97900090b0 tx=0x7f9790004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:10.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a17fa700 1 -- 172.21.15.22:0/2439575970 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97900038a0 con 0x7f97a40af2a0 2023-07-06T01:20:10.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a17fa700 1 -- 172.21.15.22:0/2439575970 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979001f070 con 0x7f97a40af2a0 2023-07-06T01:20:10.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97aaf9f700 1 -- 172.21.15.22:0/2439575970 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a415d840 con 0x7f97a40af2a0 2023-07-06T01:20:10.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97a17fa700 1 -- 172.21.15.22:0/2439575970 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790003c60 con 0x7f97a40af2a0 2023-07-06T01:20:10.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.201+0000 7f97aaf9f700 1 -- 172.21.15.22:0/2439575970 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a40b3b70 con 0x7f97a40af2a0 2023-07-06T01:20:10.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.202+0000 7f97aaf9f700 1 -- 172.21.15.22:0/2439575970 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a415e000 con 0x7f97a40af2a0 2023-07-06T01:20:10.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.203+0000 7f97a17fa700 1 -- 172.21.15.22:0/2439575970 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f979001a430 con 0x7f97a40af2a0 2023-07-06T01:20:10.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.203+0000 7f97a17fa700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f978806ace0 0x7f978806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.203+0000 7f97a17fa700 1 -- 172.21.15.22:0/2439575970 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9790021070 con 0x7f97a40af2a0 2023-07-06T01:20:10.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.203+0000 7f97a8d3b700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f978806ace0 0x7f978806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.204+0000 7f97a8d3b700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f978806ace0 0x7f978806d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9798005910 tx=0x7f9798009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:10.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.208+0000 7f97a17fa700 1 -- 172.21.15.22:0/2439575970 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9790056080 con 0x7f97a40af2a0 2023-07-06T01:20:10.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.405+0000 7f97aaf9f700 1 -- 172.21.15.22:0/2439575970 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97a40744b0 con 0x7f97a40af2a0 2023-07-06T01:20:10.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.405+0000 7f97a17fa700 1 -- 172.21.15.22:0/2439575970 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97900150a0 con 0x7f97a40af2a0 2023-07-06T01:20:10.407 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-07-06T01:20:10.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.407+0000 7f97aaf9f700 1 -- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f978806ace0 msgr2=0x7f978806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.407+0000 7f97aaf9f700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f978806ace0 0x7f978806d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9798005910 tx=0x7f9798009040 comp rx=0 tx=0).stop 2023-07-06T01:20:10.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.407+0000 7f97aaf9f700 1 -- 172.21.15.22:0/2439575970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97a40af2a0 msgr2=0x7f97a41f1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.407+0000 7f97aaf9f700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97a40af2a0 0x7f97a41f1970 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f97900090b0 tx=0x7f9790004750 comp rx=0 tx=0).stop 2023-07-06T01:20:10.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.408+0000 7f97aaf9f700 1 -- 172.21.15.22:0/2439575970 shutdown_connections 2023-07-06T01:20:10.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.408+0000 7f97aaf9f700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f97a40cbc10 0x7f97a415e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.408+0000 7f97aaf9f700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f978806ace0 0x7f978806d190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.408+0000 7f97aaf9f700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97a406a710 0x7f97a40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.408+0000 7f97aaf9f700 1 --2- 172.21.15.22:0/2439575970 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97a40af2a0 0x7f97a41f1970 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.408+0000 7f97aaf9f700 1 -- 172.21.15.22:0/2439575970 >> 172.21.15.22:0/2439575970 conn(0x7f97a40b1890 msgr2=0x7f97a40cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:10.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.408+0000 7f97aaf9f700 1 -- 172.21.15.22:0/2439575970 shutdown_connections 2023-07-06T01:20:10.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.408+0000 7f97aaf9f700 1 -- 172.21.15.22:0/2439575970 wait complete. 2023-07-06T01:20:10.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:20:10.545 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.544+0000 7f4c95c0e700 1 Processor -- start 2023-07-06T01:20:10.546 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.544+0000 7f4c95c0e700 1 -- start start 2023-07-06T01:20:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.544+0000 7f4c95c0e700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f4c900ce860 0x7f4c900cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.544+0000 7f4c95c0e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c900cbae0 0x7f4c900cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.547 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.544+0000 7f4c95c0e700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c900af270 con 0x7f4c9015a710 2023-07-06T01:20:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.544+0000 7f4c95c0e700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c900cf210 con 0x7f4c900ce860 2023-07-06T01:20:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.544+0000 7f4c95c0e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900af550 con 0x7f4c900cbae0 2023-07-06T01:20:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.545+0000 7f4c8ffff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c900cbae0 0x7f4c900cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.545+0000 7f4c8effd700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f4c900ce860 0x7f4c900cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.545+0000 7f4c8ffff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c900cbae0 0x7f4c900cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:58044/0 (socket says 172.21.15.22:58044) 2023-07-06T01:20:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.545+0000 7f4c8ffff700 1 -- 172.21.15.22:0/3679890919 learned_addr learned my addr 172.21.15.22:0/3679890919 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:10.548 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.545+0000 7f4c8ffff700 1 -- 172.21.15.22:0/3679890919 >> v2:172.21.15.103:3300/0 conn(0x7f4c900ce860 msgr2=0x7f4c900cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.545+0000 7f4c8ffff700 1 --2- 172.21.15.22:0/3679890919 >> v2:172.21.15.103:3300/0 conn(0x7f4c900ce860 0x7f4c900cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.545+0000 7f4c8ffff700 1 -- 172.21.15.22:0/3679890919 >> v1:172.21.15.103:6789/0 conn(0x7f4c9015a710 legacy=0x7f4c900ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.545+0000 7f4c8ffff700 1 -- 172.21.15.22:0/3679890919 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900af830 con 0x7f4c900cbae0 2023-07-06T01:20:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.545+0000 7f4c8effd700 1 --2- 172.21.15.22:0/3679890919 >> v2:172.21.15.103:3300/0 conn(0x7f4c900ce860 0x7f4c900cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.545+0000 7f4c8ffff700 1 --2- 172.21.15.22:0/3679890919 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c900cbae0 0x7f4c900cbf60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009e50 tx=0x7f4c8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f4a8e905282e1ac server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.546+0000 7f4c8e7fc700 1 -- 172.21.15.22:0/3679890919 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800e070 con 0x7f4c900cbae0 2023-07-06T01:20:10.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.546+0000 7f4c8e7fc700 1 -- 172.21.15.22:0/3679890919 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8800b910 con 0x7f4c900cbae0 2023-07-06T01:20:10.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.546+0000 7f4c8e7fc700 1 -- 172.21.15.22:0/3679890919 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800bcc0 con 0x7f4c900cbae0 2023-07-06T01:20:10.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.546+0000 7f4c95c0e700 1 -- 172.21.15.22:0/3679890919 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c900cbae0 msgr2=0x7f4c900cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.546+0000 7f4c95c0e700 1 --2- 172.21.15.22:0/3679890919 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c900cbae0 0x7f4c900cbf60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009e50 tx=0x7f4c8800b330 comp rx=0 tx=0).stop 2023-07-06T01:20:10.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.546+0000 7f4c95c0e700 1 -- 172.21.15.22:0/3679890919 shutdown_connections 2023-07-06T01:20:10.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.546+0000 7f4c95c0e700 1 --2- 172.21.15.22:0/3679890919 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c900cbae0 0x7f4c900cbf60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.546+0000 7f4c95c0e700 1 --2- 172.21.15.22:0/3679890919 >> v2:172.21.15.103:3300/0 conn(0x7f4c900ce860 0x7f4c900cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.546+0000 7f4c95c0e700 1 -- 172.21.15.22:0/3679890919 >> 172.21.15.22:0/3679890919 conn(0x7f4c900999f0 msgr2=0x7f4c90099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.546+0000 7f4c95c0e700 1 -- 172.21.15.22:0/3679890919 shutdown_connections 2023-07-06T01:20:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.546+0000 7f4c95c0e700 1 -- 172.21.15.22:0/3679890919 wait complete. 2023-07-06T01:20:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.548+0000 7f4c95c0e700 1 Processor -- start 2023-07-06T01:20:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.548+0000 7f4c95c0e700 1 -- start start 2023-07-06T01:20:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c95c0e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c900cbae0 0x7f4c90146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c95c0e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c900ce860 0x7f4c90158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c95c0e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c9015a710 0x7f4c901460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c95c0e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c901593f0 con 0x7f4c9015a710 2023-07-06T01:20:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c95c0e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900b4d70 con 0x7f4c900ce860 2023-07-06T01:20:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c95c0e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900b5050 con 0x7f4c900cbae0 2023-07-06T01:20:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c8ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c9015a710 0x7f4c901460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c8ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c9015a710 0x7f4c901460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46910/0 (socket says 172.21.15.22:46910) 2023-07-06T01:20:10.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c8ffff700 1 -- 172.21.15.22:0/1011788277 learned_addr learned my addr 172.21.15.22:0/1011788277 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c8effd700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c900ce860 0x7f4c90158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c8f7fe700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c900cbae0 0x7f4c90146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c8ffff700 1 -- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c900cbae0 msgr2=0x7f4c90146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c8ffff700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c900cbae0 0x7f4c90146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c8ffff700 1 -- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c900ce860 msgr2=0x7f4c90158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c8ffff700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c900ce860 0x7f4c90158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.549+0000 7f4c8ffff700 1 -- 172.21.15.22:0/1011788277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c840094a0 con 0x7f4c9015a710 2023-07-06T01:20:10.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.550+0000 7f4c8ffff700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c9015a710 0x7f4c901460a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4c900af690 tx=0x7f4c88004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.550+0000 7f4c77fff700 1 -- 172.21.15.22:0/1011788277 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88003ea0 con 0x7f4c9015a710 2023-07-06T01:20:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.550+0000 7f4c77fff700 1 -- 172.21.15.22:0/1011788277 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c880099b0 con 0x7f4c9015a710 2023-07-06T01:20:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.550+0000 7f4c95c0e700 1 -- 172.21.15.22:0/1011788277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c88009b30 con 0x7f4c9015a710 2023-07-06T01:20:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.550+0000 7f4c77fff700 1 -- 172.21.15.22:0/1011788277 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8801b670 con 0x7f4c9015a710 2023-07-06T01:20:10.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.550+0000 7f4c95c0e700 1 -- 172.21.15.22:0/1011788277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c900b5610 con 0x7f4c9015a710 2023-07-06T01:20:10.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.551+0000 7f4c95c0e700 1 -- 172.21.15.22:0/1011788277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c900744b0 con 0x7f4c9015a710 2023-07-06T01:20:10.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.551+0000 7f4c77fff700 1 -- 172.21.15.22:0/1011788277 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c8801b7d0 con 0x7f4c9015a710 2023-07-06T01:20:10.557 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.552+0000 7f4c77fff700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4c7c06ace0 0x7f4c7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.552+0000 7f4c77fff700 1 -- 172.21.15.22:0/1011788277 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c8808aa60 con 0x7f4c9015a710 2023-07-06T01:20:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.552+0000 7f4c8f7fe700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4c7c06ace0 0x7f4c7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.552+0000 7f4c8f7fe700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4c7c06ace0 0x7f4c7c06d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c800016c0 tx=0x7f4c80009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:10.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.556+0000 7f4c77fff700 1 -- 172.21.15.22:0/1011788277 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4c88055b10 con 0x7f4c9015a710 2023-07-06T01:20:10.803 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.802+0000 7f4c95c0e700 1 -- 172.21.15.22:0/1011788277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c900b6b80 con 0x7f4c9015a710 2023-07-06T01:20:10.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.802+0000 7f4c77fff700 1 -- 172.21.15.22:0/1011788277 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4671 (secure 0 0 0) 0x7f4c88059130 con 0x7f4c9015a710 2023-07-06T01:20:10.804 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:10.804 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/2496681609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2496681609},{"type":"v1","addr":"172.21.15.22:6833","nonce":2496681609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2891594418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2891594418},{"type":"v1","addr":"172.21.15.22:6835","nonce":2891594418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/4097705589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4097705589},{"type":"v1","addr":"172.21.15.103:6836","nonce":4097705589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T01:19:25.042193+0000","modified":"2023-07-06T01:19:28.649724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6837/4015300665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":4015300665},{"type":"v1","addr":"172.21.15.103:6837","nonce":4015300665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T01:20:10.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.804+0000 7f4c95c0e700 1 -- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4c7c06ace0 msgr2=0x7f4c7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.804+0000 7f4c95c0e700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4c7c06ace0 0x7f4c7c06d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c800016c0 tx=0x7f4c80009040 comp rx=0 tx=0).stop 2023-07-06T01:20:10.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.804+0000 7f4c95c0e700 1 -- 172.21.15.22:0/1011788277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c9015a710 msgr2=0x7f4c901460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.804+0000 7f4c95c0e700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c9015a710 0x7f4c901460a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4c900af690 tx=0x7f4c88004ad0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.804+0000 7f4c95c0e700 1 -- 172.21.15.22:0/1011788277 shutdown_connections 2023-07-06T01:20:10.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.804+0000 7f4c95c0e700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c9015a710 0x7f4c901460a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.804+0000 7f4c95c0e700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c900cbae0 0x7f4c90146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.804+0000 7f4c95c0e700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4c7c06ace0 0x7f4c7c06d190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.804+0000 7f4c95c0e700 1 --2- 172.21.15.22:0/1011788277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c900ce860 0x7f4c90158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.804+0000 7f4c95c0e700 1 -- 172.21.15.22:0/1011788277 >> 172.21.15.22:0/1011788277 conn(0x7f4c900999f0 msgr2=0x7f4c900ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:10.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.804+0000 7f4c95c0e700 1 -- 172.21.15.22:0/1011788277 shutdown_connections 2023-07-06T01:20:10.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.805+0000 7f4c95c0e700 1 -- 172.21.15.22:0/1011788277 wait complete. 2023-07-06T01:20:10.808 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2023-07-06T01:20:10.818 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.941+0000 7f740b112700 1 Processor -- start 2023-07-06T01:20:10.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.941+0000 7f740b112700 1 -- start start 2023-07-06T01:20:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.942+0000 7f740b112700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74040ce860 0x7f74040cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.942+0000 7f740b112700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f74040cbae0 0x7f74040cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.942+0000 7f740b112700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74040af270 con 0x7f740415a710 2023-07-06T01:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.942+0000 7f740b112700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74040cf210 con 0x7f74040cbae0 2023-07-06T01:20:10.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.942+0000 7f740b112700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74040af550 con 0x7f74040ce860 2023-07-06T01:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.942+0000 7f7403fff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74040ce860 0x7f74040cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.942+0000 7f74096af700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f74040cbae0 0x7f74040cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.942+0000 7f7403fff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74040ce860 0x7f74040cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:58056/0 (socket says 172.21.15.22:58056) 2023-07-06T01:20:10.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.942+0000 7f7403fff700 1 -- 172.21.15.22:0/2744816795 learned_addr learned my addr 172.21.15.22:0/2744816795 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.943+0000 7f7403fff700 1 -- 172.21.15.22:0/2744816795 >> v2:172.21.15.103:3300/0 conn(0x7f74040cbae0 msgr2=0x7f74040cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.943+0000 7f7403fff700 1 --2- 172.21.15.22:0/2744816795 >> v2:172.21.15.103:3300/0 conn(0x7f74040cbae0 0x7f74040cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.943+0000 7f7403fff700 1 -- 172.21.15.22:0/2744816795 >> v1:172.21.15.103:6789/0 conn(0x7f740415a710 legacy=0x7f74040ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.943+0000 7f7403fff700 1 -- 172.21.15.22:0/2744816795 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74040af830 con 0x7f74040ce860 2023-07-06T01:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.943+0000 7f7403fff700 1 --2- 172.21.15.22:0/2744816795 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74040ce860 0x7f74040cecd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f73f00098f0 tx=0x7f73f0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a71e9266f3c3bf63 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.943+0000 7f74037fe700 1 -- 172.21.15.22:0/2744816795 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0011830 con 0x7f74040ce860 2023-07-06T01:20:10.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.943+0000 7f74037fe700 1 -- 172.21.15.22:0/2744816795 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f0011990 con 0x7f74040ce860 2023-07-06T01:20:10.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.943+0000 7f74037fe700 1 -- 172.21.15.22:0/2744816795 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0011d20 con 0x7f74040ce860 2023-07-06T01:20:10.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.943+0000 7f740b112700 1 -- 172.21.15.22:0/2744816795 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74040ce860 msgr2=0x7f74040cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.943+0000 7f740b112700 1 --2- 172.21.15.22:0/2744816795 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74040ce860 0x7f74040cecd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f73f00098f0 tx=0x7f73f0009c00 comp rx=0 tx=0).stop 2023-07-06T01:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.943+0000 7f740b112700 1 -- 172.21.15.22:0/2744816795 shutdown_connections 2023-07-06T01:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.944+0000 7f740b112700 1 --2- 172.21.15.22:0/2744816795 >> v2:172.21.15.103:3300/0 conn(0x7f74040cbae0 0x7f74040cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.944+0000 7f740b112700 1 --2- 172.21.15.22:0/2744816795 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74040ce860 0x7f74040cecd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.944+0000 7f740b112700 1 -- 172.21.15.22:0/2744816795 >> 172.21.15.22:0/2744816795 conn(0x7f74040999f0 msgr2=0x7f7404099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:10.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.944+0000 7f740b112700 1 -- 172.21.15.22:0/2744816795 shutdown_connections 2023-07-06T01:20:10.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.944+0000 7f740b112700 1 -- 172.21.15.22:0/2744816795 wait complete. 2023-07-06T01:20:10.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.945+0000 7f740b112700 1 Processor -- start 2023-07-06T01:20:10.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.945+0000 7f740b112700 1 -- start start 2023-07-06T01:20:10.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f740b112700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74040cbae0 0x7f74040cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f740b112700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74040ce860 0x7f74040cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f740b112700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f740415a710 0x7f74040c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f740b112700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74040c9660 con 0x7f74040cbae0 2023-07-06T01:20:10.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f740b112700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74040c8a50 con 0x7f74040ce860 2023-07-06T01:20:10.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f740b112700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74040c8d30 con 0x7f740415a710 2023-07-06T01:20:10.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f7408eae700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74040cbae0 0x7f74040cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f7408eae700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74040cbae0 0x7f74040cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46922/0 (socket says 172.21.15.22:46922) 2023-07-06T01:20:10.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f7408eae700 1 -- 172.21.15.22:0/2665248024 learned_addr learned my addr 172.21.15.22:0/2665248024 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:10.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f7403fff700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74040ce860 0x7f74040cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f74096af700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f740415a710 0x7f74040c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f7408eae700 1 -- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f740415a710 msgr2=0x7f74040c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f7408eae700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f740415a710 0x7f74040c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f7408eae700 1 -- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74040ce860 msgr2=0x7f74040cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:10.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f7408eae700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74040ce860 0x7f74040cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:10.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f7408eae700 1 -- 172.21.15.22:0/2665248024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0009550 con 0x7f74040cbae0 2023-07-06T01:20:10.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f7408eae700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74040cbae0 0x7f74040cd550 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f73f800c450 tx=0x7f73f800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:10.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f74017fa700 1 -- 172.21.15.22:0/2665248024 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f80052d0 con 0x7f74040cbae0 2023-07-06T01:20:10.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.946+0000 7f74017fa700 1 -- 172.21.15.22:0/2665248024 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f8005430 con 0x7f74040cbae0 2023-07-06T01:20:10.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.947+0000 7f740b112700 1 -- 172.21.15.22:0/2665248024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74040c9010 con 0x7f74040cbae0 2023-07-06T01:20:10.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.947+0000 7f74017fa700 1 -- 172.21.15.22:0/2665248024 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f80038c0 con 0x7f74040cbae0 2023-07-06T01:20:10.955 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.947+0000 7f740b112700 1 -- 172.21.15.22:0/2665248024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74040c47c0 con 0x7f74040cbae0 2023-07-06T01:20:10.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.947+0000 7f740b112700 1 -- 172.21.15.22:0/2665248024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74040744b0 con 0x7f74040cbae0 2023-07-06T01:20:10.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.948+0000 7f74017fa700 1 -- 172.21.15.22:0/2665248024 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f73f8003a20 con 0x7f74040cbae0 2023-07-06T01:20:10.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.948+0000 7f74017fa700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f73e806ace0 0x7f73e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:10.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.948+0000 7f74017fa700 1 -- 172.21.15.22:0/2665248024 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f73f80887a0 con 0x7f74040cbae0 2023-07-06T01:20:10.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.949+0000 7f7403fff700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f73e806ace0 0x7f73e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:10.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.949+0000 7f7403fff700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f73e806ace0 0x7f73e806d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009c40 tx=0x7f73f0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:10.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:10.953+0000 7f74017fa700 1 -- 172.21.15.22:0/2665248024 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f73f8053850 con 0x7f74040cbae0 2023-07-06T01:20:11.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.150+0000 7f740b112700 1 -- 172.21.15.22:0/2665248024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f740406e2f0 con 0x7f74040cbae0 2023-07-06T01:20:11.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.151+0000 7f74017fa700 1 -- 172.21.15.22:0/2665248024 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18209 (secure 0 0 0) 0x7f73f800ce60 con 0x7f74040cbae0 2023-07-06T01:20:11.153 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:11.154 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:19:26.627157+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:20:11.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.154+0000 7f740b112700 1 -- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f73e806ace0 msgr2=0x7f73e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.154+0000 7f740b112700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f73e806ace0 0x7f73e806d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009c40 tx=0x7f73f0008030 comp rx=0 tx=0).stop 2023-07-06T01:20:11.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.154+0000 7f740b112700 1 -- 172.21.15.22:0/2665248024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74040cbae0 msgr2=0x7f74040cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.154+0000 7f740b112700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74040cbae0 0x7f74040cd550 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f73f800c450 tx=0x7f73f800c760 comp rx=0 tx=0).stop 2023-07-06T01:20:11.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.154+0000 7f740b112700 1 -- 172.21.15.22:0/2665248024 shutdown_connections 2023-07-06T01:20:11.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.154+0000 7f740b112700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f740415a710 0x7f74040c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.154+0000 7f740b112700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f74040cbae0 0x7f74040cd550 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.154+0000 7f740b112700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f73e806ace0 0x7f73e806d190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.154+0000 7f740b112700 1 --2- 172.21.15.22:0/2665248024 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74040ce860 0x7f74040cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.154+0000 7f740b112700 1 -- 172.21.15.22:0/2665248024 >> 172.21.15.22:0/2665248024 conn(0x7f74040999f0 msgr2=0x7f74040ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.154+0000 7f740b112700 1 -- 172.21.15.22:0/2665248024 shutdown_connections 2023-07-06T01:20:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.154+0000 7f740b112700 1 -- 172.21.15.22:0/2665248024 wait complete. 2023-07-06T01:20:11.168 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T01:20:11.169 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:20:11.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.292+0000 7f710d9ba700 1 Processor -- start 2023-07-06T01:20:11.294 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.292+0000 7f710d9ba700 1 -- start start 2023-07-06T01:20:11.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.293+0000 7f710d9ba700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71080cfc30 0x7f710806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.293+0000 7f710d9ba700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71080ce690 con 0x7f710806b650 2023-07-06T01:20:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.293+0000 7f710d9ba700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710806ba90 con 0x7f710815c750 2023-07-06T01:20:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.293+0000 7f710d9ba700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71080d06f0 con 0x7f71080cfc30 2023-07-06T01:20:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.293+0000 7f71077fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f710806b650 0x7f71080cb2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37678/0 (socket says 172.21.15.22:37678) 2023-07-06T01:20:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.293+0000 7f71077fe700 1 -- 172.21.15.22:0/3332453815 learned_addr learned my addr 172.21.15.22:0/3332453815 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f71067fc700 1 --2- 172.21.15.22:0/3332453815 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71080cfc30 0x7f710806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859139734 0 0) 0x7f71080ce690 con 0x7f710806b650 2023-07-06T01:20:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70f00035b0 con 0x7f710806b650 2023-07-06T01:20:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2735780357 0 0) 0x7f70f00035b0 con 0x7f710806b650 2023-07-06T01:20:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71080cfc30 msgr2=0x7f710806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 --2- 172.21.15.22:0/3332453815 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71080cfc30 0x7f710806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 >> v1:172.21.15.103:6789/0 conn(0x7f710815c750 legacy=0x7f71080cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71080ceb90 con 0x7f710806b650 2023-07-06T01:20:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f71000019f0 con 0x7f710806b650 2023-07-06T01:20:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 >> v1:172.21.15.22:6789/0 conn(0x7f710806b650 legacy=0x7f71080cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 --2- 172.21.15.22:0/3332453815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f0003a30 0x7f70f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 --2- 172.21.15.22:0/3332453815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70f00043d0 0x7f70f0004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 --2- 172.21.15.22:0/3332453815 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71080cfc30 0x7f70f00050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71080d06f0 con 0x7f70f00043d0 2023-07-06T01:20:11.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f000ba10 con 0x7f70f0003a30 2023-07-06T01:20:11.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f000bbd0 con 0x7f71080cfc30 2023-07-06T01:20:11.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f71067fc700 1 --2- 172.21.15.22:0/3332453815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70f00043d0 0x7f70f0004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.300 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f7106ffd700 1 --2- 172.21.15.22:0/3332453815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f0003a30 0x7f70f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f71077fe700 1 --2- 172.21.15.22:0/3332453815 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71080cfc30 0x7f70f00050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f71067fc700 1 -- 172.21.15.22:0/3332453815 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71080cfc30 msgr2=0x7f70f00050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:11.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f71067fc700 1 --2- 172.21.15.22:0/3332453815 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71080cfc30 0x7f70f00050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f71067fc700 1 -- 172.21.15.22:0/3332453815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f0003a30 msgr2=0x7f70f0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:11.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f71067fc700 1 --2- 172.21.15.22:0/3332453815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f0003a30 0x7f70f0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.294+0000 7f71067fc700 1 -- 172.21.15.22:0/3332453815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00035b0 con 0x7f70f00043d0 2023-07-06T01:20:11.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.295+0000 7f71067fc700 1 --2- 172.21.15.22:0/3332453815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70f00043d0 0x7f70f0004890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009b40 tx=0x7f70f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138a991592669669 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:11.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.295+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f800e6c0 con 0x7f70f00043d0 2023-07-06T01:20:11.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.295+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f800e820 con 0x7f70f00043d0 2023-07-06T01:20:11.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.295+0000 7f7105ffb700 1 -- 172.21.15.22:0/3332453815 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f800ebd0 con 0x7f70f00043d0 2023-07-06T01:20:11.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.295+0000 7f710d9ba700 1 -- 172.21.15.22:0/3332453815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70f00043d0 msgr2=0x7f70f0004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.295+0000 7f710d9ba700 1 --2- 172.21.15.22:0/3332453815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70f00043d0 0x7f70f0004890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009b40 tx=0x7f70f0005880 comp rx=0 tx=0).stop 2023-07-06T01:20:11.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.295+0000 7f71077fe700 1 -- 172.21.15.22:0/3332453815 reap_dead start 2023-07-06T01:20:11.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.295+0000 7f710d9ba700 1 -- 172.21.15.22:0/3332453815 shutdown_connections 2023-07-06T01:20:11.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.295+0000 7f710d9ba700 1 -- 172.21.15.22:0/3332453815 >> 172.21.15.22:0/3332453815 conn(0x7f71080999f0 msgr2=0x7f7108099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:11.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.295+0000 7f710d9ba700 1 -- 172.21.15.22:0/3332453815 shutdown_connections 2023-07-06T01:20:11.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.295+0000 7f710d9ba700 1 -- 172.21.15.22:0/3332453815 wait complete. 2023-07-06T01:20:11.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.297+0000 7f710d9ba700 1 Processor -- start 2023-07-06T01:20:11.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.297+0000 7f710d9ba700 1 -- start start 2023-07-06T01:20:11.305 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f710d9ba700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f00043d0 0x7f71081f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f710d9ba700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f710809bcc0 0x7f710809de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f710d9ba700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f710809c130 0x7f710809e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f710d9ba700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710809ac20 con 0x7f710809c130 2023-07-06T01:20:11.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f710d9ba700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710809af00 con 0x7f70f00043d0 2023-07-06T01:20:11.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f710d9ba700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710809b1e0 con 0x7f710809bcc0 2023-07-06T01:20:11.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f71077fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f710809c130 0x7f710809e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f71067fc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f710809bcc0 0x7f710809de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f71077fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f710809c130 0x7f710809e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46946/0 (socket says 172.21.15.22:46946) 2023-07-06T01:20:11.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f7106ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f00043d0 0x7f71081f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f71077fe700 1 -- 172.21.15.22:0/2157508411 learned_addr learned my addr 172.21.15.22:0/2157508411 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:11.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f71067fc700 1 -- 172.21.15.22:0/2157508411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f00043d0 msgr2=0x7f71081f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f71067fc700 1 --2- 172.21.15.22:0/2157508411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f00043d0 0x7f71081f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f71067fc700 1 -- 172.21.15.22:0/2157508411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f710809c130 msgr2=0x7f710809e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f71067fc700 1 --2- 172.21.15.22:0/2157508411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f710809c130 0x7f710809e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f71067fc700 1 -- 172.21.15.22:0/2157508411 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710809b4c0 con 0x7f710809bcc0 2023-07-06T01:20:11.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f71077fe700 1 --2- 172.21.15.22:0/2157508411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f710809c130 0x7f710809e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:11.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.298+0000 7f71067fc700 1 --2- 172.21.15.22:0/2157508411 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f710809bcc0 0x7f710809de70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f70f8019000 tx=0x7f70f80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:11.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.299+0000 7f70effff700 1 -- 172.21.15.22:0/2157508411 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f800e6c0 con 0x7f710809bcc0 2023-07-06T01:20:11.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.299+0000 7f70effff700 1 -- 172.21.15.22:0/2157508411 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f8013cb0 con 0x7f710809bcc0 2023-07-06T01:20:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.299+0000 7f710d9ba700 1 -- 172.21.15.22:0/2157508411 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710809b7a0 con 0x7f710809bcc0 2023-07-06T01:20:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.299+0000 7f70effff700 1 -- 172.21.15.22:0/2157508411 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8006e60 con 0x7f710809bcc0 2023-07-06T01:20:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.299+0000 7f710d9ba700 1 -- 172.21.15.22:0/2157508411 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71081f5470 con 0x7f710809bcc0 2023-07-06T01:20:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.299+0000 7f710d9ba700 1 -- 172.21.15.22:0/2157508411 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710809bb10 con 0x7f710809bcc0 2023-07-06T01:20:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.300+0000 7f70effff700 1 -- 172.21.15.22:0/2157508411 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70f8028690 con 0x7f710809bcc0 2023-07-06T01:20:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.300+0000 7f70effff700 1 --2- 172.21.15.22:0/2157508411 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f70f406ace0 0x7f70f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.301+0000 7f70effff700 1 -- 172.21.15.22:0/2157508411 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70f808ac20 con 0x7f710809bcc0 2023-07-06T01:20:11.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.301+0000 7f7106ffd700 1 --2- 172.21.15.22:0/2157508411 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f70f406ace0 0x7f70f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.301+0000 7f7106ffd700 1 --2- 172.21.15.22:0/2157508411 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f70f406ace0 0x7f70f406d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0011a0 tx=0x7f70fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:11.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.306+0000 7f70effff700 1 -- 172.21.15.22:0/2157508411 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f70f8055e80 con 0x7f710809bcc0 2023-07-06T01:20:11.550 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.548+0000 7f710d9ba700 1 -- 172.21.15.22:0/2157508411 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f710809b930 con 0x7f710809bcc0 2023-07-06T01:20:11.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.549+0000 7f70effff700 1 -- 172.21.15.22:0/2157508411 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4671 (secure 0 0 0) 0x7f70f8017090 con 0x7f710809bcc0 2023-07-06T01:20:11.551 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:11.551 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/2496681609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2496681609},{"type":"v1","addr":"172.21.15.22:6833","nonce":2496681609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2891594418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2891594418},{"type":"v1","addr":"172.21.15.22:6835","nonce":2891594418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/4097705589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4097705589},{"type":"v1","addr":"172.21.15.103:6836","nonce":4097705589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T01:19:25.042193+0000","modified":"2023-07-06T01:19:28.649724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6837/4015300665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":4015300665},{"type":"v1","addr":"172.21.15.103:6837","nonce":4015300665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T01:20:11.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.551+0000 7f710d9ba700 1 -- 172.21.15.22:0/2157508411 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f70f406ace0 msgr2=0x7f70f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.551+0000 7f710d9ba700 1 --2- 172.21.15.22:0/2157508411 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f70f406ace0 0x7f70f406d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0011a0 tx=0x7f70fc009040 comp rx=0 tx=0).stop 2023-07-06T01:20:11.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.551+0000 7f710d9ba700 1 -- 172.21.15.22:0/2157508411 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f710809bcc0 msgr2=0x7f710809de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.551+0000 7f710d9ba700 1 --2- 172.21.15.22:0/2157508411 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f710809bcc0 0x7f710809de70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f70f8019000 tx=0x7f70f80047c0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.551+0000 7f710d9ba700 1 -- 172.21.15.22:0/2157508411 shutdown_connections 2023-07-06T01:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.551+0000 7f710d9ba700 1 --2- 172.21.15.22:0/2157508411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f710809c130 0x7f710809e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.551+0000 7f710d9ba700 1 --2- 172.21.15.22:0/2157508411 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f70f406ace0 0x7f70f406d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.551+0000 7f710d9ba700 1 --2- 172.21.15.22:0/2157508411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70f00043d0 0x7f71081f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.551+0000 7f710d9ba700 1 --2- 172.21.15.22:0/2157508411 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f710809bcc0 0x7f710809de70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.551+0000 7f710d9ba700 1 -- 172.21.15.22:0/2157508411 >> 172.21.15.22:0/2157508411 conn(0x7f71080999f0 msgr2=0x7f710815b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:11.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.551+0000 7f710d9ba700 1 -- 172.21.15.22:0/2157508411 shutdown_connections 2023-07-06T01:20:11.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.552+0000 7f710d9ba700 1 -- 172.21.15.22:0/2157508411 wait complete. 2023-07-06T01:20:11.555 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2023-07-06T01:20:11.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:20:11.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.688+0000 7fab4fea9700 1 Processor -- start 2023-07-06T01:20:11.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.688+0000 7fab4fea9700 1 -- start start 2023-07-06T01:20:11.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.689+0000 7fab4fea9700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab480ccd70 0x7fab4815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.689+0000 7fab4fea9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fab4806d040 0x7fab4806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.689+0000 7fab4fea9700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab480cf230 con 0x7fab48159360 2023-07-06T01:20:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.689+0000 7fab4fea9700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4815df20 con 0x7fab4806d040 2023-07-06T01:20:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.689+0000 7fab4fea9700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab480ce620 con 0x7fab480ccd70 2023-07-06T01:20:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.689+0000 7fab4dc45700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fab48159360 0x7fab480cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37694/0 (socket says 172.21.15.22:37694) 2023-07-06T01:20:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.689+0000 7fab4e446700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fab4806d040 0x7fab4806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.689+0000 7fab4dc45700 1 -- 172.21.15.22:0/296210150 learned_addr learned my addr 172.21.15.22:0/296210150 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:11.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.689+0000 7fab4d444700 1 --2- 172.21.15.22:0/296210150 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab480ccd70 0x7fab4815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4e446700 1 -- 172.21.15.22:0/296210150 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab480ccd70 msgr2=0x7fab4815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:11.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4e446700 1 --2- 172.21.15.22:0/296210150 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab480ccd70 0x7fab4815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4e446700 1 -- 172.21.15.22:0/296210150 >> v1:172.21.15.22:6789/0 conn(0x7fab48159360 legacy=0x7fab480cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4cc43700 1 -- 172.21.15.22:0/296210150 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800276447 0 0) 0x7fab480cf230 con 0x7fab48159360 2023-07-06T01:20:11.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4e446700 1 -- 172.21.15.22:0/296210150 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab480ce900 con 0x7fab4806d040 2023-07-06T01:20:11.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4d444700 1 --2- 172.21.15.22:0/296210150 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab480ccd70 0x7fab4815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:11.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4e446700 1 --2- 172.21.15.22:0/296210150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab4806d040 0x7fab4806d4a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fab40009d90 tx=0x7fab4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f389cbefecb13a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:11.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4cc43700 1 -- 172.21.15.22:0/296210150 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4000e070 con 0x7fab4806d040 2023-07-06T01:20:11.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4cc43700 1 -- 172.21.15.22:0/296210150 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4000bbd0 con 0x7fab4806d040 2023-07-06T01:20:11.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4cc43700 1 -- 172.21.15.22:0/296210150 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40012430 con 0x7fab4806d040 2023-07-06T01:20:11.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4fea9700 1 -- 172.21.15.22:0/296210150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab4806d040 msgr2=0x7fab4806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4fea9700 1 --2- 172.21.15.22:0/296210150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab4806d040 0x7fab4806d4a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fab40009d90 tx=0x7fab4000b330 comp rx=0 tx=0).stop 2023-07-06T01:20:11.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4fea9700 1 -- 172.21.15.22:0/296210150 shutdown_connections 2023-07-06T01:20:11.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4fea9700 1 --2- 172.21.15.22:0/296210150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab4806d040 0x7fab4806d4a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.690+0000 7fab4fea9700 1 --2- 172.21.15.22:0/296210150 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab480ccd70 0x7fab4815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.691+0000 7fab4fea9700 1 -- 172.21.15.22:0/296210150 >> 172.21.15.22:0/296210150 conn(0x7fab480b1890 msgr2=0x7fab480b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:11.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.691+0000 7fab4fea9700 1 -- 172.21.15.22:0/296210150 shutdown_connections 2023-07-06T01:20:11.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.691+0000 7fab4fea9700 1 -- 172.21.15.22:0/296210150 wait complete. 2023-07-06T01:20:11.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.692+0000 7fab4fea9700 1 Processor -- start 2023-07-06T01:20:11.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.692+0000 7fab4fea9700 1 -- start start 2023-07-06T01:20:11.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4fea9700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab4806d040 0x7fab480afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4fea9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab480ccd70 0x7fab480aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4fea9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab48159360 0x7fab48070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4fea9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480b0740 con 0x7fab48159360 2023-07-06T01:20:11.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4fea9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480b0460 con 0x7fab480ccd70 2023-07-06T01:20:11.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4fea9700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab481f5a20 con 0x7fab4806d040 2023-07-06T01:20:11.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4e446700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab48159360 0x7fab48070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4e446700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab48159360 0x7fab48070920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46960/0 (socket says 172.21.15.22:46960) 2023-07-06T01:20:11.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4e446700 1 -- 172.21.15.22:0/107723177 learned_addr learned my addr 172.21.15.22:0/107723177 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:11.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4dc45700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab4806d040 0x7fab480afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4d444700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab480ccd70 0x7fab480aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4e446700 1 -- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab4806d040 msgr2=0x7fab480afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4e446700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab4806d040 0x7fab480afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4e446700 1 -- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab480ccd70 msgr2=0x7fab480aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4e446700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab480ccd70 0x7fab480aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4e446700 1 -- 172.21.15.22:0/107723177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab40009a70 con 0x7fab48159360 2023-07-06T01:20:11.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4dc45700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab4806d040 0x7fab480afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:11.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4e446700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab48159360 0x7fab48070920 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fab48071d00 tx=0x7fab40004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:11.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab3e7fc700 1 -- 172.21.15.22:0/107723177 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40012430 con 0x7fab48159360 2023-07-06T01:20:11.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab3e7fc700 1 -- 172.21.15.22:0/107723177 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab40020070 con 0x7fab48159360 2023-07-06T01:20:11.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4fea9700 1 -- 172.21.15.22:0/107723177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab481f5d00 con 0x7fab48159360 2023-07-06T01:20:11.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab3e7fc700 1 -- 172.21.15.22:0/107723177 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40004280 con 0x7fab48159360 2023-07-06T01:20:11.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.693+0000 7fab4fea9700 1 -- 172.21.15.22:0/107723177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab480af530 con 0x7fab48159360 2023-07-06T01:20:11.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.694+0000 7fab4fea9700 1 -- 172.21.15.22:0/107723177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab481f5880 con 0x7fab48159360 2023-07-06T01:20:11.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.695+0000 7fab3e7fc700 1 -- 172.21.15.22:0/107723177 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fab400043e0 con 0x7fab48159360 2023-07-06T01:20:11.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.695+0000 7fab3e7fc700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fab2c06ace0 0x7fab2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:11.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.695+0000 7fab3e7fc700 1 -- 172.21.15.22:0/107723177 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fab40022070 con 0x7fab48159360 2023-07-06T01:20:11.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.695+0000 7fab4dc45700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fab2c06ace0 0x7fab2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:11.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.696+0000 7fab4dc45700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fab2c06ace0 0x7fab2c06d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fab38000f60 tx=0x7fab38007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:11.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.700+0000 7fab3e7fc700 1 -- 172.21.15.22:0/107723177 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fab481f5880 con 0x7fab48159360 2023-07-06T01:20:11.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.941+0000 7fab4fea9700 1 -- 172.21.15.22:0/107723177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab480cf6a0 con 0x7fab48159360 2023-07-06T01:20:11.943 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.941+0000 7fab3e7fc700 1 -- 172.21.15.22:0/107723177 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4671 (secure 0 0 0) 0x7fab480cf6a0 con 0x7fab48159360 2023-07-06T01:20:11.943 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:11.944 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6833/2496681609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2496681609},{"type":"v1","addr":"172.21.15.22:6833","nonce":2496681609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2891594418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6834","nonce":2891594418},{"type":"v1","addr":"172.21.15.22:6835","nonce":2891594418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/4097705589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":4097705589},{"type":"v1","addr":"172.21.15.103:6836","nonce":4097705589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T01:19:25.042193+0000","modified":"2023-07-06T01:19:28.649724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.103:6837/4015300665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":4015300665},{"type":"v1","addr":"172.21.15.103:6837","nonce":4015300665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T01:20:11.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.943+0000 7fab4fea9700 1 -- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fab2c06ace0 msgr2=0x7fab2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.943+0000 7fab4fea9700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fab2c06ace0 0x7fab2c06d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fab38000f60 tx=0x7fab38007d30 comp rx=0 tx=0).stop 2023-07-06T01:20:11.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.943+0000 7fab4fea9700 1 -- 172.21.15.22:0/107723177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab48159360 msgr2=0x7fab48070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:11.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.943+0000 7fab4fea9700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab48159360 0x7fab48070920 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fab48071d00 tx=0x7fab40004ad0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.943+0000 7fab4fea9700 1 -- 172.21.15.22:0/107723177 shutdown_connections 2023-07-06T01:20:11.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.943+0000 7fab4fea9700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab48159360 0x7fab48070920 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.943+0000 7fab4fea9700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab4806d040 0x7fab480afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.943+0000 7fab4fea9700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fab2c06ace0 0x7fab2c06d190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.943+0000 7fab4fea9700 1 --2- 172.21.15.22:0/107723177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab480ccd70 0x7fab480aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:11.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.943+0000 7fab4fea9700 1 -- 172.21.15.22:0/107723177 >> 172.21.15.22:0/107723177 conn(0x7fab480b1890 msgr2=0x7fab481562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:11.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.943+0000 7fab4fea9700 1 -- 172.21.15.22:0/107723177 shutdown_connections 2023-07-06T01:20:11.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:11.943+0000 7fab4fea9700 1 -- 172.21.15.22:0/107723177 wait complete. 2023-07-06T01:20:11.947 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 7 2023-07-06T01:20:11.957 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:20:12.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.081+0000 7ffb27cb1700 1 Processor -- start 2023-07-06T01:20:12.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.081+0000 7ffb27cb1700 1 -- start start 2023-07-06T01:20:12.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb27cb1700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb200d11a0 0x7ffb200b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb27cb1700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb200cebd0 con 0x7ffb200b3b70 2023-07-06T01:20:12.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb27cb1700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb200cdfc0 con 0x7ffb200b36f0 2023-07-06T01:20:12.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb27cb1700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2015cff0 con 0x7ffb200d11a0 2023-07-06T01:20:12.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb2524c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ffb200b3b70 0x7ffb200d0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37710/0 (socket says 172.21.15.22:37710) 2023-07-06T01:20:12.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb2524c700 1 -- 172.21.15.22:0/1982759636 learned_addr learned my addr 172.21.15.22:0/1982759636 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:12.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb2624e700 1 --2- 172.21.15.22:0/1982759636 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb200d11a0 0x7ffb200b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb24a4b700 1 -- 172.21.15.22:0/1982759636 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285503278 0 0) 0x7ffb200cebd0 con 0x7ffb200b3b70 2023-07-06T01:20:12.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb24a4b700 1 -- 172.21.15.22:0/1982759636 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb0c0035b0 con 0x7ffb200b3b70 2023-07-06T01:20:12.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb24a4b700 1 -- 172.21.15.22:0/1982759636 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1603046349 0 0) 0x7ffb200cdfc0 con 0x7ffb200b36f0 2023-07-06T01:20:12.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb24a4b700 1 -- 172.21.15.22:0/1982759636 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb200cebd0 con 0x7ffb200b36f0 2023-07-06T01:20:12.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb2624e700 1 -- 172.21.15.22:0/1982759636 >> v1:172.21.15.103:6789/0 conn(0x7ffb200b36f0 legacy=0x7ffb200cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb2624e700 1 -- 172.21.15.22:0/1982759636 >> v1:172.21.15.22:6789/0 conn(0x7ffb200b3b70 legacy=0x7ffb200d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.082+0000 7ffb2624e700 1 -- 172.21.15.22:0/1982759636 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb200ce2a0 con 0x7ffb200d11a0 2023-07-06T01:20:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.083+0000 7ffb2624e700 1 --2- 172.21.15.22:0/1982759636 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb200d11a0 0x7ffb200b5da0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009dc0 tx=0x7ffb1800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=887d7f6a2c7c6a30 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.083+0000 7ffb24a4b700 1 -- 172.21.15.22:0/1982759636 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1800e070 con 0x7ffb200d11a0 2023-07-06T01:20:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.083+0000 7ffb24a4b700 1 -- 172.21.15.22:0/1982759636 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1800bb20 con 0x7ffb200d11a0 2023-07-06T01:20:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.083+0000 7ffb24a4b700 1 -- 172.21.15.22:0/1982759636 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1800be90 con 0x7ffb200d11a0 2023-07-06T01:20:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.083+0000 7ffb27cb1700 1 -- 172.21.15.22:0/1982759636 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb200d11a0 msgr2=0x7ffb200b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.083+0000 7ffb27cb1700 1 --2- 172.21.15.22:0/1982759636 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb200d11a0 0x7ffb200b5da0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009dc0 tx=0x7ffb1800b330 comp rx=0 tx=0).stop 2023-07-06T01:20:12.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.083+0000 7ffb27cb1700 1 -- 172.21.15.22:0/1982759636 shutdown_connections 2023-07-06T01:20:12.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.083+0000 7ffb27cb1700 1 --2- 172.21.15.22:0/1982759636 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb200d11a0 0x7ffb200b5da0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.083+0000 7ffb27cb1700 1 -- 172.21.15.22:0/1982759636 >> 172.21.15.22:0/1982759636 conn(0x7ffb2009c2e0 msgr2=0x7ffb200d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:12.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.083+0000 7ffb27cb1700 1 -- 172.21.15.22:0/1982759636 shutdown_connections 2023-07-06T01:20:12.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.083+0000 7ffb27cb1700 1 -- 172.21.15.22:0/1982759636 wait complete. 2023-07-06T01:20:12.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.085+0000 7ffb27cb1700 1 Processor -- start 2023-07-06T01:20:12.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.085+0000 7ffb27cb1700 1 -- start start 2023-07-06T01:20:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.085+0000 7ffb27cb1700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb200b36f0 0x7ffb200b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.085+0000 7ffb27cb1700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb200b3b70 0x7ffb201f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.085+0000 7ffb27cb1700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb200d11a0 0x7ffb201599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.085+0000 7ffb27cb1700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20152d20 con 0x7ffb200d11a0 2023-07-06T01:20:12.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.085+0000 7ffb27cb1700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2015ad40 con 0x7ffb200b3b70 2023-07-06T01:20:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.085+0000 7ffb27cb1700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20159ef0 con 0x7ffb200b36f0 2023-07-06T01:20:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb2624e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb200d11a0 0x7ffb201599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb2624e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb200d11a0 0x7ffb201599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46974/0 (socket says 172.21.15.22:46974) 2023-07-06T01:20:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb2624e700 1 -- 172.21.15.22:0/2183158282 learned_addr learned my addr 172.21.15.22:0/2183158282 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb2524c700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb200b3b70 0x7ffb201f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb25a4d700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb200b36f0 0x7ffb200b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb2624e700 1 -- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb200b36f0 msgr2=0x7ffb200b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb2624e700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb200b36f0 0x7ffb200b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb2624e700 1 -- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb200b3b70 msgr2=0x7ffb201f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb2624e700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb200b3b70 0x7ffb201f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb2624e700 1 -- 172.21.15.22:0/2183158282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb18009a70 con 0x7ffb200d11a0 2023-07-06T01:20:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb25a4d700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb200b36f0 0x7ffb200b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:12.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb2624e700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb200d11a0 0x7ffb201599b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009fd0 tx=0x7ffb18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb167fc700 1 -- 172.21.15.22:0/2183158282 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180070c0 con 0x7ffb200d11a0 2023-07-06T01:20:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb167fc700 1 -- 172.21.15.22:0/2183158282 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1801f070 con 0x7ffb200d11a0 2023-07-06T01:20:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb27cb1700 1 -- 172.21.15.22:0/2183158282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2015a1c0 con 0x7ffb200d11a0 2023-07-06T01:20:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb167fc700 1 -- 172.21.15.22:0/2183158282 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18003c20 con 0x7ffb200d11a0 2023-07-06T01:20:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.086+0000 7ffb27cb1700 1 -- 172.21.15.22:0/2183158282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2015a780 con 0x7ffb200d11a0 2023-07-06T01:20:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.087+0000 7ffb27cb1700 1 -- 172.21.15.22:0/2183158282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb200b4840 con 0x7ffb200d11a0 2023-07-06T01:20:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.088+0000 7ffb167fc700 1 -- 172.21.15.22:0/2183158282 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffb18004030 con 0x7ffb200d11a0 2023-07-06T01:20:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.088+0000 7ffb167fc700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ffb0406ace0 0x7ffb0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.088+0000 7ffb167fc700 1 -- 172.21.15.22:0/2183158282 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb1808ad40 con 0x7ffb200d11a0 2023-07-06T01:20:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.088+0000 7ffb25a4d700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ffb0406ace0 0x7ffb0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.089+0000 7ffb25a4d700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ffb0406ace0 0x7ffb0406d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c001710 tx=0x7ffb1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.093+0000 7ffb167fc700 1 -- 172.21.15.22:0/2183158282 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ffb18059870 con 0x7ffb200d11a0 2023-07-06T01:20:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.296+0000 7ffb27cb1700 1 -- 172.21.15.22:0/2183158282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb200c6a40 con 0x7ffb200d11a0 2023-07-06T01:20:12.298 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.297+0000 7ffb167fc700 1 -- 172.21.15.22:0/2183158282 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18304 (secure 0 0 0) 0x7ffb200c6a40 con 0x7ffb200d11a0 2023-07-06T01:20:12.299 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:12.299 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":22,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:20:12.273494+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:19:22.873255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T01:19:24.019088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.c","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:20:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.299+0000 7ffb27cb1700 1 -- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ffb0406ace0 msgr2=0x7ffb0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.299+0000 7ffb27cb1700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ffb0406ace0 0x7ffb0406d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c001710 tx=0x7ffb1c009040 comp rx=0 tx=0).stop 2023-07-06T01:20:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.299+0000 7ffb27cb1700 1 -- 172.21.15.22:0/2183158282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb200d11a0 msgr2=0x7ffb201599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.299+0000 7ffb27cb1700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb200d11a0 0x7ffb201599b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009fd0 tx=0x7ffb18004ad0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.299+0000 7ffb27cb1700 1 -- 172.21.15.22:0/2183158282 shutdown_connections 2023-07-06T01:20:12.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.299+0000 7ffb27cb1700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb200d11a0 0x7ffb201599b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.299+0000 7ffb27cb1700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb200b36f0 0x7ffb200b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.299+0000 7ffb27cb1700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ffb0406ace0 0x7ffb0406d190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.299+0000 7ffb27cb1700 1 --2- 172.21.15.22:0/2183158282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb200b3b70 0x7ffb201f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.299+0000 7ffb27cb1700 1 -- 172.21.15.22:0/2183158282 >> 172.21.15.22:0/2183158282 conn(0x7ffb2009c2e0 msgr2=0x7ffb2006bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:12.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.299+0000 7ffb27cb1700 1 -- 172.21.15.22:0/2183158282 shutdown_connections 2023-07-06T01:20:12.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.299+0000 7ffb27cb1700 1 -- 172.21.15.22:0/2183158282 wait complete. 2023-07-06T01:20:12.314 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T01:20:12.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.439+0000 7f9e69870700 1 Processor -- start 2023-07-06T01:20:12.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.439+0000 7f9e69870700 1 -- start start 2023-07-06T01:20:12.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.440+0000 7f9e69870700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9e6415a710 0x7f9e640ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.440+0000 7f9e69870700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e640aecb0 0x7f9e640ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.440+0000 7f9e69870700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e640af270 con 0x7f9e640ceda0 2023-07-06T01:20:12.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.440+0000 7f9e69870700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e640cf220 con 0x7f9e6415a710 2023-07-06T01:20:12.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.440+0000 7f9e69870700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640af550 con 0x7f9e640aecb0 2023-07-06T01:20:12.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.440+0000 7f9e637fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9e640ceda0 0x7f9e6415e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37726/0 (socket says 172.21.15.22:37726) 2023-07-06T01:20:12.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.440+0000 7f9e637fe700 1 -- 172.21.15.22:0/1325766472 learned_addr learned my addr 172.21.15.22:0/1325766472 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:12.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.440+0000 7f9e62ffd700 1 --2- 172.21.15.22:0/1325766472 >> v2:172.21.15.103:3300/0 conn(0x7f9e6415a710 0x7f9e640ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.440+0000 7f9e627fc700 1 --2- 172.21.15.22:0/1325766472 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e640aecb0 0x7f9e640ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576728104 0 0) 0x7f9e640af270 con 0x7f9e640ceda0 2023-07-06T01:20:12.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e4c0035b0 con 0x7f9e640ceda0 2023-07-06T01:20:12.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1382402265 0 0) 0x7f9e4c0035b0 con 0x7f9e640ceda0 2023-07-06T01:20:12.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e640aecb0 msgr2=0x7f9e640ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:12.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 --2- 172.21.15.22:0/1325766472 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e640aecb0 0x7f9e640ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 >> v2:172.21.15.103:3300/0 conn(0x7f9e6415a710 msgr2=0x7f9e640ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:12.446 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 --2- 172.21.15.22:0/1325766472 >> v2:172.21.15.103:3300/0 conn(0x7f9e6415a710 0x7f9e640ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640af830 con 0x7f9e640ceda0 2023-07-06T01:20:12.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f9e5c0019f0 con 0x7f9e640ceda0 2023-07-06T01:20:12.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 >> v1:172.21.15.22:6789/0 conn(0x7f9e640ceda0 legacy=0x7f9e6415e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 --2- 172.21.15.22:0/1325766472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e4c003a00 0x7f9e4c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 --2- 172.21.15.22:0/1325766472 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e640aecb0 0x7f9e4c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 --2- 172.21.15.22:0/1325766472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e4c004d90 0x7f9e4c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640cf220 con 0x7f9e4c004d90 2023-07-06T01:20:12.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640af550 con 0x7f9e4c003a00 2023-07-06T01:20:12.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c00ba10 con 0x7f9e640aecb0 2023-07-06T01:20:12.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e637fe700 1 --2- 172.21.15.22:0/1325766472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e4c004d90 0x7f9e4c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e627fc700 1 --2- 172.21.15.22:0/1325766472 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e640aecb0 0x7f9e4c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e62ffd700 1 --2- 172.21.15.22:0/1325766472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e4c003a00 0x7f9e4c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e637fe700 1 -- 172.21.15.22:0/1325766472 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e640aecb0 msgr2=0x7f9e4c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:12.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e637fe700 1 --2- 172.21.15.22:0/1325766472 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e640aecb0 0x7f9e4c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e637fe700 1 -- 172.21.15.22:0/1325766472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e4c003a00 msgr2=0x7f9e4c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:12.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e637fe700 1 --2- 172.21.15.22:0/1325766472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e4c003a00 0x7f9e4c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e637fe700 1 -- 172.21.15.22:0/1325766472 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c0035b0 con 0x7f9e4c004d90 2023-07-06T01:20:12.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.441+0000 7f9e637fe700 1 --2- 172.21.15.22:0/1325766472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e4c004d90 0x7f9e4c005200 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00a8d0 tx=0x7f9e4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc9454430a33434 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:12.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.442+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00a340 con 0x7f9e4c004d90 2023-07-06T01:20:12.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.442+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c012600 con 0x7f9e4c004d90 2023-07-06T01:20:12.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.442+0000 7f9e61ffb700 1 -- 172.21.15.22:0/1325766472 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0129b0 con 0x7f9e4c004d90 2023-07-06T01:20:12.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.442+0000 7f9e69870700 1 -- 172.21.15.22:0/1325766472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e4c004d90 msgr2=0x7f9e4c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.442+0000 7f9e69870700 1 --2- 172.21.15.22:0/1325766472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e4c004d90 0x7f9e4c005200 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00a8d0 tx=0x7f9e4c005880 comp rx=0 tx=0).stop 2023-07-06T01:20:12.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.442+0000 7f9e637fe700 1 -- 172.21.15.22:0/1325766472 reap_dead start 2023-07-06T01:20:12.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.442+0000 7f9e69870700 1 -- 172.21.15.22:0/1325766472 shutdown_connections 2023-07-06T01:20:12.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.442+0000 7f9e69870700 1 -- 172.21.15.22:0/1325766472 >> 172.21.15.22:0/1325766472 conn(0x7f9e640999f0 msgr2=0x7f9e64099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:12.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.442+0000 7f9e69870700 1 -- 172.21.15.22:0/1325766472 shutdown_connections 2023-07-06T01:20:12.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.442+0000 7f9e69870700 1 -- 172.21.15.22:0/1325766472 wait complete. 2023-07-06T01:20:12.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.443+0000 7f9e69870700 1 Processor -- start 2023-07-06T01:20:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.444+0000 7f9e69870700 1 -- start start 2023-07-06T01:20:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.444+0000 7f9e69870700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e64155850 0x7f9e640afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.444+0000 7f9e69870700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e64156080 0x7f9e641f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.444+0000 7f9e69870700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e640c9f30 0x7f9e640ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.444+0000 7f9e69870700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641f6b90 con 0x7f9e640c9f30 2023-07-06T01:20:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.444+0000 7f9e69870700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640ca8f0 con 0x7f9e64155850 2023-07-06T01:20:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.444+0000 7f9e69870700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640b2580 con 0x7f9e64156080 2023-07-06T01:20:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e637fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e640c9f30 0x7f9e640ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e637fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e640c9f30 0x7f9e640ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46992/0 (socket says 172.21.15.22:46992) 2023-07-06T01:20:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e637fe700 1 -- 172.21.15.22:0/3644925210 learned_addr learned my addr 172.21.15.22:0/3644925210 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e62ffd700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e64155850 0x7f9e640afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e627fc700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e64156080 0x7f9e641f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e637fe700 1 -- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e64156080 msgr2=0x7f9e641f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e637fe700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e64156080 0x7f9e641f6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e637fe700 1 -- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e64155850 msgr2=0x7f9e640afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e637fe700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e64155850 0x7f9e640afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e637fe700 1 -- 172.21.15.22:0/3644925210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640b2860 con 0x7f9e640c9f30 2023-07-06T01:20:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e62ffd700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e64155850 0x7f9e640afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e627fc700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e64156080 0x7f9e641f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:12.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e637fe700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e640c9f30 0x7f9e640ca3b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c012000 tx=0x7f9e5c001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e4bfff700 1 -- 172.21.15.22:0/3644925210 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c00a340 con 0x7f9e640c9f30 2023-07-06T01:20:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e4bfff700 1 -- 172.21.15.22:0/3644925210 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c0019c0 con 0x7f9e640c9f30 2023-07-06T01:20:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e69870700 1 -- 172.21.15.22:0/3644925210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e640b55f0 con 0x7f9e640c9f30 2023-07-06T01:20:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e4bfff700 1 -- 172.21.15.22:0/3644925210 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c027410 con 0x7f9e640c9f30 2023-07-06T01:20:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.445+0000 7f9e69870700 1 -- 172.21.15.22:0/3644925210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e640b2210 con 0x7f9e640c9f30 2023-07-06T01:20:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.446+0000 7f9e69870700 1 -- 172.21.15.22:0/3644925210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e640b5b10 con 0x7f9e640c9f30 2023-07-06T01:20:12.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.447+0000 7f9e4bfff700 1 -- 172.21.15.22:0/3644925210 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9e5c019070 con 0x7f9e640c9f30 2023-07-06T01:20:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.447+0000 7f9e4bfff700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9e5006ace0 0x7f9e5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.447+0000 7f9e4bfff700 1 -- 172.21.15.22:0/3644925210 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9e5c08a890 con 0x7f9e640c9f30 2023-07-06T01:20:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.447+0000 7f9e62ffd700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9e5006ace0 0x7f9e5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.448+0000 7f9e62ffd700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9e5006ace0 0x7f9e5006d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009c60 tx=0x7f9e58008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:12.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.451+0000 7f9e4bfff700 1 -- 172.21.15.22:0/3644925210 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9e640b5b10 con 0x7f9e640c9f30 2023-07-06T01:20:12.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.696+0000 7f9e69870700 1 -- 172.21.15.22:0/3644925210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9e640b5b10 con 0x7f9e640c9f30 2023-07-06T01:20:12.698 INFO:tasks.ceph.mon.a.smithi022.stderr:2023-07-06T01:20:12.696+0000 7ff5d4bc7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T01:20:12.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.699+0000 7f9e4bfff700 1 -- 172.21.15.22:0/3644925210 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9e640b5b10 con 0x7f9e640c9f30 2023-07-06T01:20:12.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.702+0000 7f9e69870700 1 -- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9e5006ace0 msgr2=0x7f9e5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.702+0000 7f9e69870700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9e5006ace0 0x7f9e5006d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009c60 tx=0x7f9e58008040 comp rx=0 tx=0).stop 2023-07-06T01:20:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.702+0000 7f9e69870700 1 -- 172.21.15.22:0/3644925210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e640c9f30 msgr2=0x7f9e640ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.702+0000 7f9e69870700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e640c9f30 0x7f9e640ca3b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c012000 tx=0x7f9e5c001360 comp rx=0 tx=0).stop 2023-07-06T01:20:12.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.702+0000 7f9e69870700 1 -- 172.21.15.22:0/3644925210 shutdown_connections 2023-07-06T01:20:12.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.702+0000 7f9e69870700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e640c9f30 0x7f9e640ca3b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.702+0000 7f9e69870700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9e5006ace0 0x7f9e5006d190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.702+0000 7f9e69870700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e64155850 0x7f9e640afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.702+0000 7f9e69870700 1 --2- 172.21.15.22:0/3644925210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9e64156080 0x7f9e641f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.702+0000 7f9e69870700 1 -- 172.21.15.22:0/3644925210 >> 172.21.15.22:0/3644925210 conn(0x7f9e640999f0 msgr2=0x7f9e640c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:12.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.703+0000 7f9e69870700 1 -- 172.21.15.22:0/3644925210 shutdown_connections 2023-07-06T01:20:12.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.703+0000 7f9e69870700 1 -- 172.21.15.22:0/3644925210 wait complete. 2023-07-06T01:20:12.707 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T01:20:12.718 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T01:20:12.743 INFO:tasks.ceph.mds.d.smithi103.stdout:starting mds.d at 2023-07-06T01:20:12.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.844+0000 7f5a8c1f9700 1 Processor -- start 2023-07-06T01:20:12.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.844+0000 7f5a8c1f9700 1 -- start start 2023-07-06T01:20:12.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.845+0000 7f5a8c1f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f5a8406a710 0x7f5a840ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.845+0000 7f5a8c1f9700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a840ca2b0 con 0x7f5a840cf090 2023-07-06T01:20:12.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.845+0000 7f5a8c1f9700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a840cfc90 con 0x7f5a8415cb50 2023-07-06T01:20:12.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.845+0000 7f5a8c1f9700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a840af440 con 0x7f5a8406a710 2023-07-06T01:20:12.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.845+0000 7f5a8a796700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f5a840cf090 0x7f5a84160710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37740/0 (socket says 172.21.15.22:37740) 2023-07-06T01:20:12.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.845+0000 7f5a8a796700 1 -- 172.21.15.22:0/3357137926 learned_addr learned my addr 172.21.15.22:0/3357137926 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:12.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.846+0000 7f5a89794700 1 --2- 172.21.15.22:0/3357137926 >> v2:172.21.15.103:3300/0 conn(0x7f5a8406a710 0x7f5a840ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.846+0000 7f5a88f93700 1 -- 172.21.15.22:0/3357137926 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030462703 0 0) 0x7f5a840ca2b0 con 0x7f5a840cf090 2023-07-06T01:20:12.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.846+0000 7f5a88f93700 1 -- 172.21.15.22:0/3357137926 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a700035b0 con 0x7f5a840cf090 2023-07-06T01:20:12.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.846+0000 7f5a88f93700 1 -- 172.21.15.22:0/3357137926 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391467377 0 0) 0x7f5a840cfc90 con 0x7f5a8415cb50 2023-07-06T01:20:12.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.846+0000 7f5a88f93700 1 -- 172.21.15.22:0/3357137926 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a840ca2b0 con 0x7f5a8415cb50 2023-07-06T01:20:12.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.846+0000 7f5a89794700 1 -- 172.21.15.22:0/3357137926 >> v1:172.21.15.103:6789/0 conn(0x7f5a8415cb50 legacy=0x7f5a8406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.846+0000 7f5a89794700 1 -- 172.21.15.22:0/3357137926 >> v1:172.21.15.22:6789/0 conn(0x7f5a840cf090 legacy=0x7f5a84160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.846+0000 7f5a89794700 1 -- 172.21.15.22:0/3357137926 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a840cff30 con 0x7f5a8406a710 2023-07-06T01:20:12.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.846+0000 7f5a89794700 1 --2- 172.21.15.22:0/3357137926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a8406a710 0x7f5a840ceb50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009840 tx=0x7f5a80009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be1a042a43a9c3f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:12.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.846+0000 7f5a88f93700 1 -- 172.21.15.22:0/3357137926 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80011830 con 0x7f5a8406a710 2023-07-06T01:20:12.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.846+0000 7f5a88f93700 1 -- 172.21.15.22:0/3357137926 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a80011990 con 0x7f5a8406a710 2023-07-06T01:20:12.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.846+0000 7f5a88f93700 1 -- 172.21.15.22:0/3357137926 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80011d20 con 0x7f5a8406a710 2023-07-06T01:20:12.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.847+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/3357137926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a8406a710 msgr2=0x7f5a840ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.847+0000 7f5a8c1f9700 1 --2- 172.21.15.22:0/3357137926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a8406a710 0x7f5a840ceb50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009840 tx=0x7f5a80009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:12.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.847+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/3357137926 shutdown_connections 2023-07-06T01:20:12.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.847+0000 7f5a8c1f9700 1 --2- 172.21.15.22:0/3357137926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a8406a710 0x7f5a840ceb50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.847+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/3357137926 >> 172.21.15.22:0/3357137926 conn(0x7f5a8406b110 msgr2=0x7f5a8406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:12.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.847+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/3357137926 shutdown_connections 2023-07-06T01:20:12.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.847+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/3357137926 wait complete. 2023-07-06T01:20:12.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.848+0000 7f5a8c1f9700 1 Processor -- start 2023-07-06T01:20:12.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.848+0000 7f5a8c1f9700 1 -- start start 2023-07-06T01:20:12.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8c1f9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a8406a710 0x7f5a8415df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8c1f9700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a840cf090 0x7f5a840c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8c1f9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a8415cb50 0x7f5a84147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8c1f9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8415f910 con 0x7f5a8415cb50 2023-07-06T01:20:12.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8c1f9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8415f480 con 0x7f5a8406a710 2023-07-06T01:20:12.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8c1f9700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8415e870 con 0x7f5a840cf090 2023-07-06T01:20:12.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8a796700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a8415cb50 0x7f5a84147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8a796700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a8415cb50 0x7f5a84147b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47004/0 (socket says 172.21.15.22:47004) 2023-07-06T01:20:12.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8a796700 1 -- 172.21.15.22:0/2684431794 learned_addr learned my addr 172.21.15.22:0/2684431794 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:12.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a89794700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a840cf090 0x7f5a840c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a89f95700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a8406a710 0x7f5a8415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8a796700 1 -- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a840cf090 msgr2=0x7f5a840c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8a796700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a840cf090 0x7f5a840c3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8a796700 1 -- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a8406a710 msgr2=0x7f5a8415df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:12.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8a796700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a8406a710 0x7f5a8415df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:12.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8a796700 1 -- 172.21.15.22:0/2684431794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a800094a0 con 0x7f5a8415cb50 2023-07-06T01:20:12.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.849+0000 7f5a8a796700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a8415cb50 0x7f5a84147b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00cba0 tx=0x7f5a7c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:12.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.850+0000 7f5a7a7fc700 1 -- 172.21.15.22:0/2684431794 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c00a980 con 0x7f5a8415cb50 2023-07-06T01:20:12.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.850+0000 7f5a7a7fc700 1 -- 172.21.15.22:0/2684431794 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7c00aae0 con 0x7f5a8415cb50 2023-07-06T01:20:12.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.850+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/2684431794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8415eb50 con 0x7f5a8415cb50 2023-07-06T01:20:12.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.850+0000 7f5a7a7fc700 1 -- 172.21.15.22:0/2684431794 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c00ae50 con 0x7f5a8415cb50 2023-07-06T01:20:12.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.850+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/2684431794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8415f110 con 0x7f5a8415cb50 2023-07-06T01:20:12.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.851+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/2684431794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8415f5f0 con 0x7f5a8415cb50 2023-07-06T01:20:12.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.851+0000 7f5a7a7fc700 1 -- 172.21.15.22:0/2684431794 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a7c01b3b0 con 0x7f5a8415cb50 2023-07-06T01:20:12.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.851+0000 7f5a7a7fc700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5a6806ace0 0x7f5a6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:12.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.852+0000 7f5a7a7fc700 1 -- 172.21.15.22:0/2684431794 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f5a7c089030 con 0x7f5a8415cb50 2023-07-06T01:20:12.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.852+0000 7f5a89f95700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5a6806ace0 0x7f5a6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:12.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.852+0000 7f5a89f95700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5a6806ace0 0x7f5a6806d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5a740011e0 tx=0x7f5a74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:12.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:12.856+0000 7f5a7a7fc700 1 -- 172.21.15.22:0/2684431794 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5a7c057b90 con 0x7f5a8415cb50 2023-07-06T01:20:13.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.099+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/2684431794 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5a840744b0 con 0x7f5a8415cb50 2023-07-06T01:20:13.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.700+0000 7f5a7a7fc700 1 -- 172.21.15.22:0/2684431794 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5a7c057720 con 0x7f5a8415cb50 2023-07-06T01:20:13.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.703+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5a6806ace0 msgr2=0x7f5a6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:13.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.703+0000 7f5a8c1f9700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5a6806ace0 0x7f5a6806d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5a740011e0 tx=0x7f5a74009040 comp rx=0 tx=0).stop 2023-07-06T01:20:13.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.703+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/2684431794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a8415cb50 msgr2=0x7f5a84147b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:13.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.703+0000 7f5a8c1f9700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a8415cb50 0x7f5a84147b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00cba0 tx=0x7f5a7c00ceb0 comp rx=0 tx=0).stop 2023-07-06T01:20:13.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.703+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/2684431794 shutdown_connections 2023-07-06T01:20:13.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.703+0000 7f5a8c1f9700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a8415cb50 0x7f5a84147b50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:13.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.703+0000 7f5a8c1f9700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5a6806ace0 0x7f5a6806d190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:13.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.703+0000 7f5a8c1f9700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a8406a710 0x7f5a8415df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:13.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.703+0000 7f5a8c1f9700 1 --2- 172.21.15.22:0/2684431794 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a840cf090 0x7f5a840c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:13.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.703+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/2684431794 >> 172.21.15.22:0/2684431794 conn(0x7f5a8406b110 msgr2=0x7f5a8415c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:13.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.703+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/2684431794 shutdown_connections 2023-07-06T01:20:13.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.703+0000 7f5a8c1f9700 1 -- 172.21.15.22:0/2684431794 wait complete. 2023-07-06T01:20:13.719 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T01:20:13.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.850+0000 7f786033c700 1 Processor -- start 2023-07-06T01:20:13.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.850+0000 7f786033c700 1 -- start start 2023-07-06T01:20:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.851+0000 7f786033c700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f785815a710 0x7f78580ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:13.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.851+0000 7f786033c700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78580af270 con 0x7f78580cef70 2023-07-06T01:20:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.851+0000 7f786033c700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78580af550 con 0x7f78580aecb0 2023-07-06T01:20:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.851+0000 7f786033c700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78580af830 con 0x7f785815a710 2023-07-06T01:20:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.851+0000 7f785e8d9700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f78580cef70 0x7f78580cbae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37752/0 (socket says 172.21.15.22:37752) 2023-07-06T01:20:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785e8d9700 1 -- 172.21.15.22:0/2076569125 learned_addr learned my addr 172.21.15.22:0/2076569125 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785e0d8700 1 --2- 172.21.15.22:0/2076569125 >> v2:172.21.15.103:3300/0 conn(0x7f785815a710 0x7f78580ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:13.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880403123 0 0) 0x7f78580af270 con 0x7f78580cef70 2023-07-06T01:20:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78440035b0 con 0x7f78580cef70 2023-07-06T01:20:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111176052 0 0) 0x7f78580af550 con 0x7f78580aecb0 2023-07-06T01:20:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78580af270 con 0x7f78580aecb0 2023-07-06T01:20:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3415444813 0 0) 0x7f78440035b0 con 0x7f78580cef70 2023-07-06T01:20:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 >> v2:172.21.15.103:3300/0 conn(0x7f785815a710 msgr2=0x7f78580ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 --2- 172.21.15.22:0/2076569125 >> v2:172.21.15.103:3300/0 conn(0x7f785815a710 0x7f78580ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 >> v1:172.21.15.103:6789/0 conn(0x7f78580aecb0 legacy=0x7f78580ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:13.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78580cb2e0 con 0x7f78580cef70 2023-07-06T01:20:13.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f78500019f0 con 0x7f78580cef70 2023-07-06T01:20:13.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785e0d8700 1 --2- 172.21.15.22:0/2076569125 >> v2:172.21.15.103:3300/0 conn(0x7f785815a710 0x7f78580ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:13.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 >> v1:172.21.15.22:6789/0 conn(0x7f78580cef70 legacy=0x7f78580cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:13.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 --2- 172.21.15.22:0/2076569125 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7844003b90 0x7f7844003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:13.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 --2- 172.21.15.22:0/2076569125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7844004850 0x7f7844004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:13.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 --2- 172.21.15.22:0/2076569125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7844005200 0x7f784400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:13.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78580af830 con 0x7f7844004850 2023-07-06T01:20:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784400c060 con 0x7f7844005200 2023-07-06T01:20:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784400c220 con 0x7f7844003b90 2023-07-06T01:20:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785d8d7700 1 --2- 172.21.15.22:0/2076569125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7844004850 0x7f7844004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785e0d8700 1 --2- 172.21.15.22:0/2076569125 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7844003b90 0x7f7844003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.852+0000 7f785e8d9700 1 --2- 172.21.15.22:0/2076569125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7844005200 0x7f784400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.853+0000 7f785d8d7700 1 -- 172.21.15.22:0/2076569125 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7844003b90 msgr2=0x7f7844003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.853+0000 7f785d8d7700 1 --2- 172.21.15.22:0/2076569125 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7844003b90 0x7f7844003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:13.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.853+0000 7f785d8d7700 1 -- 172.21.15.22:0/2076569125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7844005200 msgr2=0x7f784400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:13.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.853+0000 7f785d8d7700 1 --2- 172.21.15.22:0/2076569125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7844005200 0x7f784400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:13.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.853+0000 7f785e8d9700 1 -- 172.21.15.22:0/2076569125 reap_dead start 2023-07-06T01:20:13.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.853+0000 7f785d8d7700 1 -- 172.21.15.22:0/2076569125 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78440035b0 con 0x7f7844004850 2023-07-06T01:20:13.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.853+0000 7f785d8d7700 1 --2- 172.21.15.22:0/2076569125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7844004850 0x7f7844004cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7848001940 tx=0x7f7844005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc1a83ca713f5fd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:13.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.853+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848001430 con 0x7f7844004850 2023-07-06T01:20:13.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.853+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7848001d70 con 0x7f7844004850 2023-07-06T01:20:13.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.853+0000 7f785d0d6700 1 -- 172.21.15.22:0/2076569125 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848005530 con 0x7f7844004850 2023-07-06T01:20:13.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.853+0000 7f786033c700 1 -- 172.21.15.22:0/2076569125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7844004850 msgr2=0x7f7844004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:13.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.853+0000 7f786033c700 1 --2- 172.21.15.22:0/2076569125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7844004850 0x7f7844004cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7848001940 tx=0x7f7844005880 comp rx=0 tx=0).stop 2023-07-06T01:20:13.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.853+0000 7f786033c700 1 -- 172.21.15.22:0/2076569125 shutdown_connections 2023-07-06T01:20:13.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.854+0000 7f786033c700 1 --2- 172.21.15.22:0/2076569125 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7844004850 0x7f7844004cc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:13.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.854+0000 7f786033c700 1 -- 172.21.15.22:0/2076569125 >> 172.21.15.22:0/2076569125 conn(0x7f78580999f0 msgr2=0x7f7858099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:13.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.854+0000 7f786033c700 1 -- 172.21.15.22:0/2076569125 shutdown_connections 2023-07-06T01:20:13.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.854+0000 7f786033c700 1 -- 172.21.15.22:0/2076569125 wait complete. 2023-07-06T01:20:13.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.855+0000 7f786033c700 1 Processor -- start 2023-07-06T01:20:13.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.855+0000 7f786033c700 1 -- start start 2023-07-06T01:20:13.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.856+0000 7f786033c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7844004850 0x7f785815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:13.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.856+0000 7f786033c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f785815cae0 0x7f78580c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:13.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.856+0000 7f786033c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78581544a0 0x7f78580c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:13.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.856+0000 7f786033c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78580c7e00 con 0x7f7844004850 2023-07-06T01:20:13.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.856+0000 7f786033c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78580c80e0 con 0x7f78581544a0 2023-07-06T01:20:13.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.856+0000 7f786033c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78580c83c0 con 0x7f785815cae0 2023-07-06T01:20:13.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.856+0000 7f785e0d8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7844004850 0x7f785815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:13.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.856+0000 7f785e0d8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7844004850 0x7f785815c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47018/0 (socket says 172.21.15.22:47018) 2023-07-06T01:20:13.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.856+0000 7f785e0d8700 1 -- 172.21.15.22:0/64740073 learned_addr learned my addr 172.21.15.22:0/64740073 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:13.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.856+0000 7f785d8d7700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f785815cae0 0x7f78580c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:13.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.856+0000 7f785e8d9700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78581544a0 0x7f78580c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:13.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.857+0000 7f785e0d8700 1 -- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f785815cae0 msgr2=0x7f78580c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:13.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.857+0000 7f785e0d8700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f785815cae0 0x7f78580c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:13.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.857+0000 7f785e0d8700 1 -- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78581544a0 msgr2=0x7f78580c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:13.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.857+0000 7f785e0d8700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78581544a0 0x7f78580c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:13.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.857+0000 7f785e0d8700 1 -- 172.21.15.22:0/64740073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78580c86a0 con 0x7f7844004850 2023-07-06T01:20:13.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.857+0000 7f785d8d7700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f785815cae0 0x7f78580c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:13.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.857+0000 7f785e8d9700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78581544a0 0x7f78580c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:13.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.857+0000 7f785e0d8700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7844004850 0x7f785815c5a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7854011c20 tx=0x7f785400f240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:13.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.857+0000 7f784effd700 1 -- 172.21.15.22:0/64740073 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785400fb80 con 0x7f7844004850 2023-07-06T01:20:13.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.857+0000 7f784effd700 1 -- 172.21.15.22:0/64740073 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785400fce0 con 0x7f7844004850 2023-07-06T01:20:13.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.857+0000 7f786033c700 1 -- 172.21.15.22:0/64740073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78580c8980 con 0x7f7844004850 2023-07-06T01:20:13.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.857+0000 7f784effd700 1 -- 172.21.15.22:0/64740073 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854004240 con 0x7f7844004850 2023-07-06T01:20:13.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.857+0000 7f786033c700 1 -- 172.21.15.22:0/64740073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785815d9a0 con 0x7f7844004850 2023-07-06T01:20:13.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.858+0000 7f786033c700 1 -- 172.21.15.22:0/64740073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78580c8cf0 con 0x7f7844004850 2023-07-06T01:20:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.858+0000 7f784effd700 1 -- 172.21.15.22:0/64740073 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f78540043c0 con 0x7f7844004850 2023-07-06T01:20:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.859+0000 7f784effd700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f783c06ace0 0x7f783c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.859+0000 7f784effd700 1 -- 172.21.15.22:0/64740073 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f7854089b30 con 0x7f7844004850 2023-07-06T01:20:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.859+0000 7f785d8d7700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f783c06ace0 0x7f783c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.859+0000 7f785d8d7700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f783c06ace0 0x7f783c06d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7848005040 tx=0x7f784800a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:13.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:13.863+0000 7f784effd700 1 -- 172.21.15.22:0/64740073 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7854054b50 con 0x7f7844004850 2023-07-06T01:20:14.066 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.064+0000 7f786033c700 1 -- 172.21.15.22:0/64740073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f78580ae720 con 0x7f7844004850 2023-07-06T01:20:14.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.706+0000 7f784effd700 1 -- 172.21.15.22:0/64740073 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7854022020 con 0x7f7844004850 2023-07-06T01:20:14.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.711+0000 7f786033c700 1 -- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f783c06ace0 msgr2=0x7f783c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:14.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.711+0000 7f786033c700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f783c06ace0 0x7f783c06d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7848005040 tx=0x7f784800a000 comp rx=0 tx=0).stop 2023-07-06T01:20:14.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.711+0000 7f786033c700 1 -- 172.21.15.22:0/64740073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7844004850 msgr2=0x7f785815c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:14.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.711+0000 7f786033c700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7844004850 0x7f785815c5a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7854011c20 tx=0x7f785400f240 comp rx=0 tx=0).stop 2023-07-06T01:20:14.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.712+0000 7f786033c700 1 -- 172.21.15.22:0/64740073 shutdown_connections 2023-07-06T01:20:14.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.712+0000 7f786033c700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f783c06ace0 0x7f783c06d190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:14.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.712+0000 7f786033c700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78581544a0 0x7f78580c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:14.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.712+0000 7f786033c700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7844004850 0x7f785815c5a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:14.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.712+0000 7f786033c700 1 --2- 172.21.15.22:0/64740073 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f785815cae0 0x7f78580c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:14.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.712+0000 7f786033c700 1 -- 172.21.15.22:0/64740073 >> 172.21.15.22:0/64740073 conn(0x7f78580999f0 msgr2=0x7f7858099dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:14.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.712+0000 7f786033c700 1 -- 172.21.15.22:0/64740073 shutdown_connections 2023-07-06T01:20:14.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.713+0000 7f786033c700 1 -- 172.21.15.22:0/64740073 wait complete. 2023-07-06T01:20:14.718 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2023-07-06T01:20:14.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T01:20:14.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.863+0000 7f815241d700 1 Processor -- start 2023-07-06T01:20:14.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.863+0000 7f815241d700 1 -- start start 2023-07-06T01:20:14.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.864+0000 7f815241d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0cecd0 0x7f814c0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:14.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.864+0000 7f815241d700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814c0af270 con 0x7f814c15a710 2023-07-06T01:20:14.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.864+0000 7f815241d700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814c0af550 con 0x7f814c0ce860 2023-07-06T01:20:14.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.864+0000 7f815241d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0af830 con 0x7f814c0cecd0 2023-07-06T01:20:14.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.864+0000 7f814bfff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f814c15a710 0x7f814c0ae770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37770/0 (socket says 172.21.15.22:37770) 2023-07-06T01:20:14.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.864+0000 7f814bfff700 1 -- 172.21.15.22:0/1823290622 learned_addr learned my addr 172.21.15.22:0/1823290622 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:14.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f81509ba700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0cecd0 0x7f814c0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:14.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981195757 0 0) 0x7f814c0af270 con 0x7f814c15a710 2023-07-06T01:20:14.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81380035b0 con 0x7f814c15a710 2023-07-06T01:20:14.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3571071334 0 0) 0x7f81380035b0 con 0x7f814c15a710 2023-07-06T01:20:14.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0cecd0 msgr2=0x7f814c0cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:14.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0cecd0 0x7f814c0cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 >> v1:172.21.15.103:6789/0 conn(0x7f814c0ce860 legacy=0x7f814c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c0cb2e0 con 0x7f814c15a710 2023-07-06T01:20:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f813c001670 con 0x7f814c15a710 2023-07-06T01:20:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 >> v1:172.21.15.22:6789/0 conn(0x7f814c15a710 legacy=0x7f814c0ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8138003a30 0x7f8138003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81380043d0 0x7f8138004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0cecd0 0x7f81380050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:14.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0af830 con 0x7f8138003a30 2023-07-06T01:20:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813800ba10 con 0x7f81380043d0 2023-07-06T01:20:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813800bbd0 con 0x7f814c0cecd0 2023-07-06T01:20:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814bfff700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8138003a30 0x7f8138003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f81509ba700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0cecd0 0x7f81380050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814b7fe700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81380043d0 0x7f8138004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814bfff700 1 -- 172.21.15.22:0/1823290622 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0cecd0 msgr2=0x7f81380050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:14.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814bfff700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0cecd0 0x7f81380050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:14.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814bfff700 1 -- 172.21.15.22:0/1823290622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81380043d0 msgr2=0x7f8138004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:14.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814bfff700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81380043d0 0x7f8138004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:14.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814bfff700 1 -- 172.21.15.22:0/1823290622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81380035b0 con 0x7f8138003a30 2023-07-06T01:20:14.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f81509ba700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0cecd0 0x7f81380050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:14.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.865+0000 7f814b7fe700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81380043d0 0x7f8138004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:14.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.866+0000 7f814bfff700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8138003a30 0x7f8138003e00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f813c00c2f0 tx=0x7f8138005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b145dfb2013039 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:14.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.866+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c00ce30 con 0x7f8138003a30 2023-07-06T01:20:14.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.866+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c005960 con 0x7f8138003a30 2023-07-06T01:20:14.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.866+0000 7f814affd700 1 -- 172.21.15.22:0/1823290622 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c005cf0 con 0x7f8138003a30 2023-07-06T01:20:14.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.866+0000 7f815241d700 1 -- 172.21.15.22:0/1823290622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8138003a30 msgr2=0x7f8138003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:14.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.866+0000 7f815241d700 1 --2- 172.21.15.22:0/1823290622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8138003a30 0x7f8138003e00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f813c00c2f0 tx=0x7f8138005880 comp rx=0 tx=0).stop 2023-07-06T01:20:14.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.866+0000 7f81509ba700 1 -- 172.21.15.22:0/1823290622 reap_dead start 2023-07-06T01:20:14.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.866+0000 7f815241d700 1 -- 172.21.15.22:0/1823290622 shutdown_connections 2023-07-06T01:20:14.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.866+0000 7f815241d700 1 -- 172.21.15.22:0/1823290622 >> 172.21.15.22:0/1823290622 conn(0x7f814c0999f0 msgr2=0x7f814c099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:14.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.866+0000 7f815241d700 1 -- 172.21.15.22:0/1823290622 shutdown_connections 2023-07-06T01:20:14.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.866+0000 7f815241d700 1 -- 172.21.15.22:0/1823290622 wait complete. 2023-07-06T01:20:14.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.868+0000 7f815241d700 1 Processor -- start 2023-07-06T01:20:14.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.868+0000 7f815241d700 1 -- start start 2023-07-06T01:20:14.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f815241d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8138003a30 0x7f814c0b6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:14.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f815241d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f814c0c6460 0x7f814c0c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:14.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f815241d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0b1dd0 0x7f814c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:14.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f815241d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0c6e10 con 0x7f8138003a30 2023-07-06T01:20:14.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f815241d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0b2b90 con 0x7f814c0c6460 2023-07-06T01:20:14.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f815241d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0b3b50 con 0x7f814c0b1dd0 2023-07-06T01:20:14.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f814bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8138003a30 0x7f814c0b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:14.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f814bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8138003a30 0x7f814c0b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47038/0 (socket says 172.21.15.22:47038) 2023-07-06T01:20:14.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f814bfff700 1 -- 172.21.15.22:0/1197034671 learned_addr learned my addr 172.21.15.22:0/1197034671 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:14.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f814b7fe700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f814c0c6460 0x7f814c0c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:14.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f81509ba700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0b1dd0 0x7f814c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:14.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f814bfff700 1 -- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0b1dd0 msgr2=0x7f814c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:14.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f814bfff700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0b1dd0 0x7f814c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:14.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f814bfff700 1 -- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f814c0c6460 msgr2=0x7f814c0c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:14.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f814bfff700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f814c0c6460 0x7f814c0c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:14.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f814bfff700 1 -- 172.21.15.22:0/1197034671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c0b3e30 con 0x7f8138003a30 2023-07-06T01:20:14.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f81509ba700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0b1dd0 0x7f814c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:14.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f814bfff700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8138003a30 0x7f814c0b6510 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f813c00c8e0 tx=0x7f813c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:14.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f8148ff9700 1 -- 172.21.15.22:0/1197034671 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c005ce0 con 0x7f8138003a30 2023-07-06T01:20:14.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f8148ff9700 1 -- 172.21.15.22:0/1197034671 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c005e40 con 0x7f8138003a30 2023-07-06T01:20:14.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f8148ff9700 1 -- 172.21.15.22:0/1197034671 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c004a70 con 0x7f8138003a30 2023-07-06T01:20:14.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f815241d700 1 -- 172.21.15.22:0/1197034671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814c0b4110 con 0x7f8138003a30 2023-07-06T01:20:14.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.869+0000 7f815241d700 1 -- 172.21.15.22:0/1197034671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814c0b6a50 con 0x7f8138003a30 2023-07-06T01:20:14.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.870+0000 7f815241d700 1 -- 172.21.15.22:0/1197034671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814c0744b0 con 0x7f8138003a30 2023-07-06T01:20:14.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.871+0000 7f8148ff9700 1 -- 172.21.15.22:0/1197034671 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f813c004bd0 con 0x7f8138003a30 2023-07-06T01:20:14.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.871+0000 7f8148ff9700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f812c06ace0 0x7f812c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:14.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.871+0000 7f8148ff9700 1 -- 172.21.15.22:0/1197034671 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f813c08a5d0 con 0x7f8138003a30 2023-07-06T01:20:14.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.871+0000 7f814b7fe700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f812c06ace0 0x7f812c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:14.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.872+0000 7f814b7fe700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f812c06ace0 0x7f812c06d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8134000c00 tx=0x7f8134009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:14.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:14.876+0000 7f8148ff9700 1 -- 172.21.15.22:0/1197034671 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f813c0558f0 con 0x7f8138003a30 2023-07-06T01:20:15.082 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.080+0000 7f815241d700 1 -- 172.21.15.22:0/1197034671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f814c0ade20 con 0x7f8138003a30 2023-07-06T01:20:15.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.709+0000 7f8148ff9700 1 -- 172.21.15.22:0/1197034671 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f813c019310 con 0x7f8138003a30 2023-07-06T01:20:15.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.714+0000 7f815241d700 1 -- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f812c06ace0 msgr2=0x7f812c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:15.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.714+0000 7f815241d700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f812c06ace0 0x7f812c06d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8134000c00 tx=0x7f8134009040 comp rx=0 tx=0).stop 2023-07-06T01:20:15.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.714+0000 7f815241d700 1 -- 172.21.15.22:0/1197034671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8138003a30 msgr2=0x7f814c0b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:15.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.714+0000 7f815241d700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8138003a30 0x7f814c0b6510 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f813c00c8e0 tx=0x7f813c001a00 comp rx=0 tx=0).stop 2023-07-06T01:20:15.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.715+0000 7f815241d700 1 -- 172.21.15.22:0/1197034671 shutdown_connections 2023-07-06T01:20:15.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.715+0000 7f815241d700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f814c0b1dd0 0x7f814c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:15.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.715+0000 7f815241d700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8138003a30 0x7f814c0b6510 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:15.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.715+0000 7f815241d700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f812c06ace0 0x7f812c06d190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:15.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.715+0000 7f815241d700 1 --2- 172.21.15.22:0/1197034671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f814c0c6460 0x7f814c0c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:15.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.715+0000 7f815241d700 1 -- 172.21.15.22:0/1197034671 >> 172.21.15.22:0/1197034671 conn(0x7f814c0999f0 msgr2=0x7f814c0cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:15.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.716+0000 7f815241d700 1 -- 172.21.15.22:0/1197034671 shutdown_connections 2023-07-06T01:20:15.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.716+0000 7f815241d700 1 -- 172.21.15.22:0/1197034671 wait complete. 2023-07-06T01:20:15.720 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2023-07-06T01:20:15.735 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T01:20:15.736 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T01:20:15.736 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T01:20:15.736 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:20:15.737 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T01:20:15.737 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T01:20:15.737 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T01:20:15.737 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:20:15.737 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T01:20:15.738 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T01:20:15.738 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T01:20:15.738 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:20:15.738 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T01:20:15.739 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T01:20:15.739 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T01:20:15.739 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:20:15.746 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T01:20:15.746 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T01:20:15.747 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T01:20:15.748 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T01:20:15.749 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T01:20:15.749 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T01:20:15.750 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T01:20:15.750 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T01:20:15.789 INFO:tasks.ceph.mds.a:Started 2023-07-06T01:20:15.789 DEBUG:teuthology.parallel:result is None 2023-07-06T01:20:15.790 INFO:tasks.ceph.mds.c:Started 2023-07-06T01:20:15.791 DEBUG:teuthology.parallel:result is None 2023-07-06T01:20:15.791 INFO:tasks.ceph.mds.b:Started 2023-07-06T01:20:15.792 DEBUG:teuthology.parallel:result is None 2023-07-06T01:20:15.792 INFO:tasks.ceph.mds.d:Started 2023-07-06T01:20:15.793 DEBUG:teuthology.parallel:result is None 2023-07-06T01:20:15.796 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T01:20:15.878 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2023-07-06T01:20:15.878 INFO:tasks.ceph.mds.d.smithi103.stdout:starting mds.d at 2023-07-06T01:20:15.879 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2023-07-06T01:20:15.879 INFO:tasks.ceph.mds.b.smithi103.stdout:starting mds.b at 2023-07-06T01:20:15.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.931+0000 7fb196c7b700 1 Processor -- start 2023-07-06T01:20:15.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.931+0000 7fb196c7b700 1 -- start start 2023-07-06T01:20:15.935 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb196c7b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb19015a710 0x7fb1900ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:15.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb196c7b700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fb1900cef70 0x7fb1900cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:15.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb196c7b700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1900af270 con 0x7fb1900aecb0 2023-07-06T01:20:15.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb196c7b700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1900af550 con 0x7fb1900cef70 2023-07-06T01:20:15.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb196c7b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1900af830 con 0x7fb19015a710 2023-07-06T01:20:15.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18ffff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb1900aecb0 0x7fb1900ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37818/0 (socket says 172.21.15.22:37818) 2023-07-06T01:20:15.936 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18ffff700 1 -- 172.21.15.22:0/2329458462 learned_addr learned my addr 172.21.15.22:0/2329458462 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:15.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb195218700 1 --2- 172.21.15.22:0/2329458462 >> v2:172.21.15.103:3300/0 conn(0x7fb1900cef70 0x7fb1900cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:15.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb194a17700 1 --2- 172.21.15.22:0/2329458462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb19015a710 0x7fb1900ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:15.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252143539 0 0) 0x7fb1900af270 con 0x7fb1900aecb0 2023-07-06T01:20:15.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1800035b0 con 0x7fb1900aecb0 2023-07-06T01:20:15.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2283639157 0 0) 0x7fb1800035b0 con 0x7fb1900aecb0 2023-07-06T01:20:15.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb19015a710 msgr2=0x7fb1900ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:15.937 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18f7fe700 1 --2- 172.21.15.22:0/2329458462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb19015a710 0x7fb1900ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:15.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 >> v2:172.21.15.103:3300/0 conn(0x7fb1900cef70 msgr2=0x7fb1900cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:15.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18f7fe700 1 --2- 172.21.15.22:0/2329458462 >> v2:172.21.15.103:3300/0 conn(0x7fb1900cef70 0x7fb1900cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:15.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1900cb2e0 con 0x7fb1900aecb0 2023-07-06T01:20:15.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7fb17c001670 con 0x7fb1900aecb0 2023-07-06T01:20:15.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 >> v1:172.21.15.22:6789/0 conn(0x7fb1900aecb0 legacy=0x7fb1900ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:15.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18f7fe700 1 --2- 172.21.15.22:0/2329458462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb180003a00 0x7fb180003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:15.938 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18f7fe700 1 --2- 172.21.15.22:0/2329458462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb180004430 0x7fb1800048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:15.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.932+0000 7fb18f7fe700 1 --2- 172.21.15.22:0/2329458462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb180004f60 0x7fb18000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:15.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1900af550 con 0x7fb180003a00 2023-07-06T01:20:15.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1900af830 con 0x7fb180004430 2023-07-06T01:20:15.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18000bf50 con 0x7fb180004f60 2023-07-06T01:20:15.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb195218700 1 --2- 172.21.15.22:0/2329458462 >> v2:172.21.15.103:3300/0 conn(0x7fb1900cef70 0x7fb1900cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:15.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb194a17700 1 --2- 172.21.15.22:0/2329458462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb19015a710 0x7fb1900ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:15.939 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb194a17700 1 --2- 172.21.15.22:0/2329458462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb180003a00 0x7fb180003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb18ffff700 1 --2- 172.21.15.22:0/2329458462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb180004430 0x7fb1800048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb195218700 1 --2- 172.21.15.22:0/2329458462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb180004f60 0x7fb18000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb194a17700 1 -- 172.21.15.22:0/2329458462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb180004f60 msgr2=0x7fb18000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb194a17700 1 --2- 172.21.15.22:0/2329458462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb180004f60 0x7fb18000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb194a17700 1 -- 172.21.15.22:0/2329458462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb180004430 msgr2=0x7fb1800048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb194a17700 1 --2- 172.21.15.22:0/2329458462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb180004430 0x7fb1800048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:15.940 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb194a17700 1 -- 172.21.15.22:0/2329458462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1800035b0 con 0x7fb180003a00 2023-07-06T01:20:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb194a17700 1 --2- 172.21.15.22:0/2329458462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb180003a00 0x7fb180003e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb190070f30 tx=0x7fb180005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5e35559489fe2a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184004030 con 0x7fb180003a00 2023-07-06T01:20:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb184004190 con 0x7fb180003a00 2023-07-06T01:20:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.933+0000 7fb18f7fe700 1 -- 172.21.15.22:0/2329458462 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184004540 con 0x7fb180003a00 2023-07-06T01:20:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.934+0000 7fb196c7b700 1 -- 172.21.15.22:0/2329458462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb180003a00 msgr2=0x7fb180003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.934+0000 7fb196c7b700 1 --2- 172.21.15.22:0/2329458462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb180003a00 0x7fb180003e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb190070f30 tx=0x7fb180005880 comp rx=0 tx=0).stop 2023-07-06T01:20:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.934+0000 7fb195218700 1 -- 172.21.15.22:0/2329458462 reap_dead start 2023-07-06T01:20:15.941 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.934+0000 7fb196c7b700 1 -- 172.21.15.22:0/2329458462 shutdown_connections 2023-07-06T01:20:15.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.934+0000 7fb196c7b700 1 -- 172.21.15.22:0/2329458462 >> 172.21.15.22:0/2329458462 conn(0x7fb1900999f0 msgr2=0x7fb190099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:15.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.934+0000 7fb196c7b700 1 -- 172.21.15.22:0/2329458462 shutdown_connections 2023-07-06T01:20:15.942 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.934+0000 7fb196c7b700 1 -- 172.21.15.22:0/2329458462 wait complete. 2023-07-06T01:20:15.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.935+0000 7fb196c7b700 1 Processor -- start 2023-07-06T01:20:15.944 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.935+0000 7fb196c7b700 1 -- start start 2023-07-06T01:20:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb196c7b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb180003a00 0x7fb1900c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb196c7b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb19006caa0 0x7fb1900c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb196c7b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb19006e240 0x7fb19006e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb196c7b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19006b110 con 0x7fb180003a00 2023-07-06T01:20:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb196c7b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19006d1a0 con 0x7fb19006caa0 2023-07-06T01:20:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb196c7b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19006ebf0 con 0x7fb19006e240 2023-07-06T01:20:15.945 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb194a17700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb180003a00 0x7fb1900c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb194a17700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb180003a00 0x7fb1900c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47084/0 (socket says 172.21.15.22:47084) 2023-07-06T01:20:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb194a17700 1 -- 172.21.15.22:0/875739589 learned_addr learned my addr 172.21.15.22:0/875739589 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb194a17700 1 -- 172.21.15.22:0/875739589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb19006e240 msgr2=0x7fb19006e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb18ffff700 1 --2- 172.21.15.22:0/875739589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb19006caa0 0x7fb1900c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb194a17700 1 --2- 172.21.15.22:0/875739589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb19006e240 0x7fb19006e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb194a17700 1 -- 172.21.15.22:0/875739589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb19006caa0 msgr2=0x7fb1900c3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:15.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb194a17700 1 --2- 172.21.15.22:0/875739589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb19006caa0 0x7fb1900c3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb194a17700 1 -- 172.21.15.22:0/875739589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1901f60c0 con 0x7fb180003a00 2023-07-06T01:20:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.936+0000 7fb194a17700 1 --2- 172.21.15.22:0/875739589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb180003a00 0x7fb1900c7250 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb18400eb40 tx=0x7fb1840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.937+0000 7fb18d7fa700 1 -- 172.21.15.22:0/875739589 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184004030 con 0x7fb180003a00 2023-07-06T01:20:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.937+0000 7fb18d7fa700 1 -- 172.21.15.22:0/875739589 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb184003ab0 con 0x7fb180003a00 2023-07-06T01:20:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.937+0000 7fb18d7fa700 1 -- 172.21.15.22:0/875739589 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184003e40 con 0x7fb180003a00 2023-07-06T01:20:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.937+0000 7fb196c7b700 1 -- 172.21.15.22:0/875739589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19006cf10 con 0x7fb180003a00 2023-07-06T01:20:15.947 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.937+0000 7fb196c7b700 1 -- 172.21.15.22:0/875739589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1901f6550 con 0x7fb180003a00 2023-07-06T01:20:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.938+0000 7fb196c7b700 1 -- 172.21.15.22:0/875739589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1900744b0 con 0x7fb180003a00 2023-07-06T01:20:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.938+0000 7fb18d7fa700 1 -- 172.21.15.22:0/875739589 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb184013470 con 0x7fb180003a00 2023-07-06T01:20:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.938+0000 7fb18d7fa700 1 --2- 172.21.15.22:0/875739589 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb17406ace0 0x7fb17406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.939+0000 7fb18d7fa700 1 -- 172.21.15.22:0/875739589 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb18408a930 con 0x7fb180003a00 2023-07-06T01:20:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.939+0000 7fb18ffff700 1 --2- 172.21.15.22:0/875739589 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb17406ace0 0x7fb17406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:15.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.939+0000 7fb18ffff700 1 --2- 172.21.15.22:0/875739589 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb17406ace0 0x7fb17406d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb17c000c00 tx=0x7fb17c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:15.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:15.943+0000 7fb18d7fa700 1 -- 172.21.15.22:0/875739589 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb184055d70 con 0x7fb180003a00 2023-07-06T01:20:16.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:16.168+0000 7fb196c7b700 1 -- 172.21.15.22:0/875739589 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb19009a200 con 0x7fb180003a00 2023-07-06T01:20:17.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.177+0000 7fb18d7fa700 1 -- 172.21.15.22:0/875739589 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb184059390 con 0x7fb180003a00 2023-07-06T01:20:17.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.180+0000 7fb196c7b700 1 -- 172.21.15.22:0/875739589 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb17406ace0 msgr2=0x7fb17406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.180+0000 7fb196c7b700 1 --2- 172.21.15.22:0/875739589 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb17406ace0 0x7fb17406d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb17c000c00 tx=0x7fb17c009040 comp rx=0 tx=0).stop 2023-07-06T01:20:17.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.180+0000 7fb196c7b700 1 -- 172.21.15.22:0/875739589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb180003a00 msgr2=0x7fb1900c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.180+0000 7fb196c7b700 1 --2- 172.21.15.22:0/875739589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb180003a00 0x7fb1900c7250 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb18400eb40 tx=0x7fb1840047c0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.180+0000 7fb196c7b700 1 -- 172.21.15.22:0/875739589 shutdown_connections 2023-07-06T01:20:17.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.180+0000 7fb196c7b700 1 --2- 172.21.15.22:0/875739589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb19006e240 0x7fb19006e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.180+0000 7fb196c7b700 1 --2- 172.21.15.22:0/875739589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb180003a00 0x7fb1900c7250 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.180+0000 7fb196c7b700 1 --2- 172.21.15.22:0/875739589 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb17406ace0 0x7fb17406d190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.180+0000 7fb196c7b700 1 --2- 172.21.15.22:0/875739589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb19006caa0 0x7fb1900c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.180+0000 7fb196c7b700 1 -- 172.21.15.22:0/875739589 >> 172.21.15.22:0/875739589 conn(0x7fb1900999f0 msgr2=0x7fb1900cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:17.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.180+0000 7fb196c7b700 1 -- 172.21.15.22:0/875739589 shutdown_connections 2023-07-06T01:20:17.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.180+0000 7fb196c7b700 1 -- 172.21.15.22:0/875739589 wait complete. 2023-07-06T01:20:17.184 INFO:teuthology.orchestra.run.smithi022.stderr: removed all blocklist entries 2023-07-06T01:20:17.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T01:20:17.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.329+0000 7f3f34bd9700 1 Processor -- start 2023-07-06T01:20:17.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.329+0000 7f3f34bd9700 1 -- start start 2023-07-06T01:20:17.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.330+0000 7f3f34bd9700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f300cf260 0x7f3f30167a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:17.332 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.330+0000 7f3f34bd9700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f300ca2b0 con 0x7f3f3006a710 2023-07-06T01:20:17.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.330+0000 7f3f34bd9700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f300cf6e0 con 0x7f3f3015cb50 2023-07-06T01:20:17.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.330+0000 7f3f34bd9700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300af440 con 0x7f3f300cf260 2023-07-06T01:20:17.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.330+0000 7f3f2dd9b700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f3f3006a710 0x7f3f300ceb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37842/0 (socket says 172.21.15.22:37842) 2023-07-06T01:20:17.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.330+0000 7f3f2dd9b700 1 -- 172.21.15.22:0/3308630788 learned_addr learned my addr 172.21.15.22:0/3308630788 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:17.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.330+0000 7f3f2ed9d700 1 --2- 172.21.15.22:0/3308630788 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f300cf260 0x7f3f30167a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:17.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.330+0000 7f3f2d59a700 1 -- 172.21.15.22:0/3308630788 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232987709 0 0) 0x7f3f300ca2b0 con 0x7f3f3006a710 2023-07-06T01:20:17.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.330+0000 7f3f2d59a700 1 -- 172.21.15.22:0/3308630788 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f180035b0 con 0x7f3f3006a710 2023-07-06T01:20:17.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.330+0000 7f3f2d59a700 1 -- 172.21.15.22:0/3308630788 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448809679 0 0) 0x7f3f300cf6e0 con 0x7f3f3015cb50 2023-07-06T01:20:17.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.331+0000 7f3f2d59a700 1 -- 172.21.15.22:0/3308630788 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f300ca2b0 con 0x7f3f3015cb50 2023-07-06T01:20:17.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.331+0000 7f3f2d59a700 1 -- 172.21.15.22:0/3308630788 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1846277836 0 0) 0x7f3f180035b0 con 0x7f3f3006a710 2023-07-06T01:20:17.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.331+0000 7f3f2ed9d700 1 -- 172.21.15.22:0/3308630788 >> v1:172.21.15.103:6789/0 conn(0x7f3f3015cb50 legacy=0x7f3f3006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.331+0000 7f3f2ed9d700 1 -- 172.21.15.22:0/3308630788 >> v1:172.21.15.22:6789/0 conn(0x7f3f3006a710 legacy=0x7f3f300ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.331+0000 7f3f2ed9d700 1 -- 172.21.15.22:0/3308630788 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f300cfc90 con 0x7f3f300cf260 2023-07-06T01:20:17.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.331+0000 7f3f2ed9d700 1 --2- 172.21.15.22:0/3308630788 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f300cf260 0x7f3f30167a40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009dc0 tx=0x7f3f2800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22247ed28b75559 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.331+0000 7f3f2d59a700 1 -- 172.21.15.22:0/3308630788 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2800e070 con 0x7f3f300cf260 2023-07-06T01:20:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.331+0000 7f3f2d59a700 1 -- 172.21.15.22:0/3308630788 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f28003a20 con 0x7f3f300cf260 2023-07-06T01:20:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.331+0000 7f3f2d59a700 1 -- 172.21.15.22:0/3308630788 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20001670 con 0x7f3f300cf260 2023-07-06T01:20:17.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.331+0000 7f3f34bd9700 1 -- 172.21.15.22:0/3308630788 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f300cf260 msgr2=0x7f3f30167a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.332+0000 7f3f34bd9700 1 --2- 172.21.15.22:0/3308630788 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f300cf260 0x7f3f30167a40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009dc0 tx=0x7f3f2800b330 comp rx=0 tx=0).stop 2023-07-06T01:20:17.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.332+0000 7f3f34bd9700 1 -- 172.21.15.22:0/3308630788 shutdown_connections 2023-07-06T01:20:17.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.332+0000 7f3f34bd9700 1 --2- 172.21.15.22:0/3308630788 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f300cf260 0x7f3f30167a40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.332+0000 7f3f34bd9700 1 -- 172.21.15.22:0/3308630788 >> 172.21.15.22:0/3308630788 conn(0x7f3f3006b110 msgr2=0x7f3f3006b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:17.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.332+0000 7f3f34bd9700 1 -- 172.21.15.22:0/3308630788 shutdown_connections 2023-07-06T01:20:17.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.332+0000 7f3f34bd9700 1 -- 172.21.15.22:0/3308630788 wait complete. 2023-07-06T01:20:17.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.333+0000 7f3f34bd9700 1 Processor -- start 2023-07-06T01:20:17.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.333+0000 7f3f34bd9700 1 -- start start 2023-07-06T01:20:17.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f34bd9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f3006a710 0x7f3f3015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:17.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f34bd9700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f300cf260 0x7f3f300c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:17.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f34bd9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f3015cb50 0x7f3f300c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:17.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f34bd9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3015dba0 con 0x7f3f3006a710 2023-07-06T01:20:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f34bd9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300b4a40 con 0x7f3f3015cb50 2023-07-06T01:20:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f34bd9700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300b3e30 con 0x7f3f300cf260 2023-07-06T01:20:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f2e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f3006a710 0x7f3f3015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f2e59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f3006a710 0x7f3f3015b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47102/0 (socket says 172.21.15.22:47102) 2023-07-06T01:20:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f2e59c700 1 -- 172.21.15.22:0/2795829207 learned_addr learned my addr 172.21.15.22:0/2795829207 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f2dd9b700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f300cf260 0x7f3f300c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f2ed9d700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f3015cb50 0x7f3f300c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f2e59c700 1 -- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f300cf260 msgr2=0x7f3f300c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f2e59c700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f300cf260 0x7f3f300c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f2e59c700 1 -- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f3015cb50 msgr2=0x7f3f300c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f2e59c700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f3015cb50 0x7f3f300c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f2e59c700 1 -- 172.21.15.22:0/2795829207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f28009a70 con 0x7f3f3006a710 2023-07-06T01:20:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f2e59c700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f3006a710 0x7f3f3015b2b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400c410 tx=0x7f3f2400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.334+0000 7f3f1effd700 1 -- 172.21.15.22:0/2795829207 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240052d0 con 0x7f3f3006a710 2023-07-06T01:20:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.335+0000 7f3f1effd700 1 -- 172.21.15.22:0/2795829207 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f24005450 con 0x7f3f3006a710 2023-07-06T01:20:17.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.335+0000 7f3f34bd9700 1 -- 172.21.15.22:0/2795829207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f300b4110 con 0x7f3f3006a710 2023-07-06T01:20:17.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.335+0000 7f3f1effd700 1 -- 172.21.15.22:0/2795829207 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240038c0 con 0x7f3f3006a710 2023-07-06T01:20:17.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.335+0000 7f3f34bd9700 1 -- 172.21.15.22:0/2795829207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f300b46d0 con 0x7f3f3006a710 2023-07-06T01:20:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.336+0000 7f3f34bd9700 1 -- 172.21.15.22:0/2795829207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f300744b0 con 0x7f3f3006a710 2023-07-06T01:20:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.336+0000 7f3f1effd700 1 -- 172.21.15.22:0/2795829207 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f24003a20 con 0x7f3f3006a710 2023-07-06T01:20:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.337+0000 7f3f1effd700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3f1006ace0 0x7f3f1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.337+0000 7f3f1effd700 1 -- 172.21.15.22:0/2795829207 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3f24088600 con 0x7f3f3006a710 2023-07-06T01:20:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.337+0000 7f3f2dd9b700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3f1006ace0 0x7f3f1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.337+0000 7f3f2dd9b700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3f1006ace0 0x7f3f1006d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3f300c9f60 tx=0x7f3f20001590 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:17.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.341+0000 7f3f1effd700 1 -- 172.21.15.22:0/2795829207 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3f24053a00 con 0x7f3f3006a710 2023-07-06T01:20:17.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.619+0000 7f3f34bd9700 1 -- 172.21.15.22:0/2795829207 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3f300ccc30 con 0x7f3f3006a710 2023-07-06T01:20:17.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.619+0000 7f3f1effd700 1 -- 172.21.15.22:0/2795829207 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f3f24057020 con 0x7f3f3006a710 2023-07-06T01:20:17.621 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:17.621 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:20:17.622 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_dump": [ 2023-07-06T01:20:17.622 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.622 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.a", 2023-07-06T01:20:17.622 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBqFqZkV6iICRAA1MCk3dMsqNkCf72IGijUcw==", 2023-07-06T01:20:17.622 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.622 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-07-06T01:20:17.622 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2023-07-06T01:20:17.623 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2023-07-06T01:20:17.623 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.623 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.623 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.623 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.623 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.b", 2023-07-06T01:20:17.623 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBqFqZkA0TfFRAArj+u5qjLMmHyy1NJ09DUyQ==", 2023-07-06T01:20:17.624 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.624 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-07-06T01:20:17.624 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2023-07-06T01:20:17.624 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2023-07-06T01:20:17.624 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.624 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.624 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.625 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.625 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.c", 2023-07-06T01:20:17.625 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBqFqZkAC7dEBAAwFbKmtddOUgk+zFNHeXARA==", 2023-07-06T01:20:17.625 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.625 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-07-06T01:20:17.625 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2023-07-06T01:20:17.625 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2023-07-06T01:20:17.625 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.626 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.626 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.626 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.626 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.d", 2023-07-06T01:20:17.626 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBqFqZkWCkbHRAAjWrgjyK0BdXxv4ThUrFtYg==", 2023-07-06T01:20:17.626 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.626 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-07-06T01:20:17.627 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2023-07-06T01:20:17.627 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2023-07-06T01:20:17.627 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.627 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.627 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.627 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.627 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.0", 2023-07-06T01:20:17.628 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBsFqZkm0BeExAAI1tQQ8r8sP59nJtjyvb0JQ==", 2023-07-06T01:20:17.628 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.628 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-07-06T01:20:17.628 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-07-06T01:20:17.628 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.628 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.628 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.629 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.629 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.1", 2023-07-06T01:20:17.629 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBvFqZk7FLJEhAAnu8BLLGSpyL1ZF+YyXbRmA==", 2023-07-06T01:20:17.629 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.629 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-07-06T01:20:17.629 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-07-06T01:20:17.629 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.629 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.630 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.630 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.2", 2023-07-06T01:20:17.630 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQByFqZkowUtFRAAVagTRRRKP99vm4ax6e5OzA==", 2023-07-06T01:20:17.630 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.630 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-07-06T01:20:17.630 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-07-06T01:20:17.631 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.631 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.631 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.631 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.3", 2023-07-06T01:20:17.631 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQB1FqZkx0pyFRAAjYI2ThVYi4um9goY+b4hWQ==", 2023-07-06T01:20:17.631 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.632 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-07-06T01:20:17.632 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-07-06T01:20:17.632 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.632 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.632 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.632 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.632 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.4", 2023-07-06T01:20:17.632 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQB7FqZka3s3GxAAypfzEphyjKJ1tHo9Yl9nMQ==", 2023-07-06T01:20:17.633 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.633 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-07-06T01:20:17.633 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-07-06T01:20:17.633 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.633 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.633 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.633 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.5", 2023-07-06T01:20:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQB+FqZk1eOUGhAAnxp7kVV8jkttPyEFjrHVTQ==", 2023-07-06T01:20:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-07-06T01:20:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-07-06T01:20:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.6", 2023-07-06T01:20:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCBFqZk27ehGRAAROQLW6hs1IZY5GlwkdhwGA==", 2023-07-06T01:20:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-07-06T01:20:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-07-06T01:20:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.636 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.636 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.636 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.636 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.7", 2023-07-06T01:20:17.636 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCEFqZk4z0wGRAAWJ0U5aaSyQ8vOdbI0rk5+g==", 2023-07-06T01:20:17.636 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.636 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2023-07-06T01:20:17.637 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2023-07-06T01:20:17.637 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.637 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.637 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.637 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.637 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.0", 2023-07-06T01:20:17.637 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBqFqZkwHh1IRAAx8BVLni20wF8yL0mJjooBQ==", 2023-07-06T01:20:17.637 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.638 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-07-06T01:20:17.638 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2023-07-06T01:20:17.638 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2023-07-06T01:20:17.638 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2023-07-06T01:20:17.638 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.638 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.638 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.639 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.1", 2023-07-06T01:20:17.639 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBqFqZkb9FaLBAATGB1K++aTCBst5UwjU16zw==", 2023-07-06T01:20:17.639 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.639 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-07-06T01:20:17.639 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2023-07-06T01:20:17.639 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2023-07-06T01:20:17.640 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2023-07-06T01:20:17.640 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.640 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.640 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.640 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.2", 2023-07-06T01:20:17.640 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBqFqZky+nZJBAABn14hveejk0SaouG1DVesw==", 2023-07-06T01:20:17.640 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.641 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-07-06T01:20:17.641 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2023-07-06T01:20:17.641 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2023-07-06T01:20:17.641 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2023-07-06T01:20:17.641 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.641 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.641 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.3", 2023-07-06T01:20:17.642 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBqFqZkKlurLxAAgM2osFvBfK2mlvgsxyuPyQ==", 2023-07-06T01:20:17.642 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.642 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2023-07-06T01:20:17.642 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2023-07-06T01:20:17.642 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2023-07-06T01:20:17.642 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2023-07-06T01:20:17.642 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.643 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.643 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.643 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.admin", 2023-07-06T01:20:17.643 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBpFqZkAvl9JxAAvDENFU+XESId1Ds8YTHCVw==", 2023-07-06T01:20:17.643 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.643 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2023-07-06T01:20:17.643 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2023-07-06T01:20:17.644 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2023-07-06T01:20:17.644 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.644 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.644 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.644 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-mds", 2023-07-06T01:20:17.644 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCKFqZkqPnkFxAAduUr9mIgDQ4rPnSYRZyLbQ==", 2023-07-06T01:20:17.644 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.645 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-mds" 2023-07-06T01:20:17.645 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.645 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.645 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-mgr", 2023-07-06T01:20:17.645 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCKFqZk5gTlFxAAZOGF1pj3q7T+eo7eoeFoEg==", 2023-07-06T01:20:17.645 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.645 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-mgr" 2023-07-06T01:20:17.646 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.646 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.646 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.646 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-osd", 2023-07-06T01:20:17.646 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCKFqZk9w3lFxAAGQNpZTglcIoU9GlvXn7ZJg==", 2023-07-06T01:20:17.646 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.646 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-osd" 2023-07-06T01:20:17.646 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.647 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.647 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rbd", 2023-07-06T01:20:17.647 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCKFqZk7xXlFxAAmTy5Fbu2USx2ToGEafKNkg==", 2023-07-06T01:20:17.647 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.647 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rbd" 2023-07-06T01:20:17.647 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.648 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.648 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-07-06T01:20:17.648 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCKFqZkCR7lFxAATbmfTubLgJOCLT22OTuvUg==", 2023-07-06T01:20:17.648 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.648 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-07-06T01:20:17.648 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.649 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rgw", 2023-07-06T01:20:17.649 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQCKFqZkHCblFxAAkybH7ZX/i3FEolySUI4mFA==", 2023-07-06T01:20:17.649 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.649 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rgw" 2023-07-06T01:20:17.649 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.649 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.650 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.x", 2023-07-06T01:20:17.650 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBqFqZkL8wRBhAAbGoxdH55UXARG50fMSNM9w==", 2023-07-06T01:20:17.650 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.650 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2023-07-06T01:20:17.650 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2023-07-06T01:20:17.650 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.650 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:17.651 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:17.651 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.y", 2023-07-06T01:20:17.651 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQBqFqZkxDmKARAAFtK3jIDy4CK63pSxXdfYgA==", 2023-07-06T01:20:17.651 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2023-07-06T01:20:17.651 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2023-07-06T01:20:17.651 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2023-07-06T01:20:17.651 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2023-07-06T01:20:17.652 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.652 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:17.652 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2023-07-06T01:20:17.652 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:20:17.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.621+0000 7f3f34bd9700 1 -- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3f1006ace0 msgr2=0x7f3f1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.621+0000 7f3f34bd9700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3f1006ace0 0x7f3f1006d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3f300c9f60 tx=0x7f3f20001590 comp rx=0 tx=0).stop 2023-07-06T01:20:17.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.621+0000 7f3f34bd9700 1 -- 172.21.15.22:0/2795829207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f3006a710 msgr2=0x7f3f3015b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.621+0000 7f3f34bd9700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f3006a710 0x7f3f3015b2b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400c410 tx=0x7f3f2400c7d0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.621+0000 7f3f34bd9700 1 -- 172.21.15.22:0/2795829207 shutdown_connections 2023-07-06T01:20:17.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.621+0000 7f3f34bd9700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3f1006ace0 0x7f3f1006d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.621+0000 7f3f34bd9700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f3015cb50 0x7f3f300c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.621+0000 7f3f34bd9700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f3006a710 0x7f3f3015b2b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.621+0000 7f3f34bd9700 1 --2- 172.21.15.22:0/2795829207 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f300cf260 0x7f3f300c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.621+0000 7f3f34bd9700 1 -- 172.21.15.22:0/2795829207 >> 172.21.15.22:0/2795829207 conn(0x7f3f3006b110 msgr2=0x7f3f3015beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:17.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.621+0000 7f3f34bd9700 1 -- 172.21.15.22:0/2795829207 shutdown_connections 2023-07-06T01:20:17.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.621+0000 7f3f34bd9700 1 -- 172.21.15.22:0/2795829207 wait complete. 2023-07-06T01:20:17.655 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-07-06T01:20:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.781+0000 7f63cdb9b700 1 Processor -- start 2023-07-06T01:20:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.781+0000 7f63cdb9b700 1 -- start start 2023-07-06T01:20:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63cdb9b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63c815df70 0x7f63c80cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63cdb9b700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c80ce550 con 0x7f63c80d0130 2023-07-06T01:20:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63cdb9b700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c80cdb60 con 0x7f63c8159870 2023-07-06T01:20:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63cdb9b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c80d05b0 con 0x7f63c815df70 2023-07-06T01:20:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63c7fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f63c80d0130 0x7f63c80ca360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37858/0 (socket says 172.21.15.22:37858) 2023-07-06T01:20:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63c7fff700 1 -- 172.21.15.22:0/5721755 learned_addr learned my addr 172.21.15.22:0/5721755 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211704956 0 0) 0x7f63c80ce550 con 0x7f63c80d0130 2023-07-06T01:20:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63c6ffd700 1 --2- 172.21.15.22:0/5721755 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63c815df70 0x7f63c80cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63b00035b0 con 0x7f63c80d0130 2023-07-06T01:20:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 10112744 0 0) 0x7f63b00035b0 con 0x7f63c80d0130 2023-07-06T01:20:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63c815df70 msgr2=0x7f63c80cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63c67fc700 1 --2- 172.21.15.22:0/5721755 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63c815df70 0x7f63c80cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 >> v1:172.21.15.103:6789/0 conn(0x7f63c8159870 legacy=0x7f63c80715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c815d490 con 0x7f63c80d0130 2023-07-06T01:20:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.782+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f63c00019f0 con 0x7f63c80d0130 2023-07-06T01:20:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 >> v1:172.21.15.22:6789/0 conn(0x7f63c80d0130 legacy=0x7f63c80ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c67fc700 1 --2- 172.21.15.22:0/5721755 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63c815df70 0x7f63b0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c67fc700 1 --2- 172.21.15.22:0/5721755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63b0003f70 0x7f63b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c67fc700 1 --2- 172.21.15.22:0/5721755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63b0004d90 0x7f63b0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c80d05b0 con 0x7f63b0003f70 2023-07-06T01:20:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b000ba40 con 0x7f63b0004d90 2023-07-06T01:20:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b000bc00 con 0x7f63c815df70 2023-07-06T01:20:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c6ffd700 1 --2- 172.21.15.22:0/5721755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63b0003f70 0x7f63b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c77fe700 1 --2- 172.21.15.22:0/5721755 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63c815df70 0x7f63b0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c7fff700 1 --2- 172.21.15.22:0/5721755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63b0004d90 0x7f63b0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c6ffd700 1 -- 172.21.15.22:0/5721755 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63c815df70 msgr2=0x7f63b0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c6ffd700 1 --2- 172.21.15.22:0/5721755 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63c815df70 0x7f63b0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c6ffd700 1 -- 172.21.15.22:0/5721755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63b0004d90 msgr2=0x7f63b0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:17.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c6ffd700 1 --2- 172.21.15.22:0/5721755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63b0004d90 0x7f63b0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c6ffd700 1 -- 172.21.15.22:0/5721755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b00035b0 con 0x7f63b0003f70 2023-07-06T01:20:17.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c7fff700 1 --2- 172.21.15.22:0/5721755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63b0004d90 0x7f63b0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:17.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c6ffd700 1 --2- 172.21.15.22:0/5721755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63b0003f70 0x7f63b0004850 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009b60 tx=0x7f63b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2f1150fb783572 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:17.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00e6c0 con 0x7f63b0003f70 2023-07-06T01:20:17.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc00e820 con 0x7f63b0003f70 2023-07-06T01:20:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.783+0000 7f63c67fc700 1 -- 172.21.15.22:0/5721755 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00eb90 con 0x7f63b0003f70 2023-07-06T01:20:17.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.784+0000 7f63cdb9b700 1 -- 172.21.15.22:0/5721755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63b0003f70 msgr2=0x7f63b0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.784+0000 7f63cdb9b700 1 --2- 172.21.15.22:0/5721755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63b0003f70 0x7f63b0004850 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009b60 tx=0x7f63b0005880 comp rx=0 tx=0).stop 2023-07-06T01:20:17.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.784+0000 7f63c7fff700 1 -- 172.21.15.22:0/5721755 reap_dead start 2023-07-06T01:20:17.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.784+0000 7f63cdb9b700 1 -- 172.21.15.22:0/5721755 shutdown_connections 2023-07-06T01:20:17.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.784+0000 7f63cdb9b700 1 -- 172.21.15.22:0/5721755 >> 172.21.15.22:0/5721755 conn(0x7f63c806b110 msgr2=0x7f63c806b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:17.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.784+0000 7f63cdb9b700 1 -- 172.21.15.22:0/5721755 shutdown_connections 2023-07-06T01:20:17.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.784+0000 7f63cdb9b700 1 -- 172.21.15.22:0/5721755 wait complete. 2023-07-06T01:20:17.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.785+0000 7f63cdb9b700 1 Processor -- start 2023-07-06T01:20:17.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.786+0000 7f63cdb9b700 1 -- start start 2023-07-06T01:20:17.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63cdb9b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63b0003f70 0x7f63c814d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:17.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63cdb9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63c814f5b0 0x7f63c814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:17.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63cdb9b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63c806a000 0x7f63c806a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:17.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63cdb9b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c80b2b40 con 0x7f63c814f5b0 2023-07-06T01:20:17.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63cdb9b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c8150320 con 0x7f63b0003f70 2023-07-06T01:20:17.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63cdb9b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c8150600 con 0x7f63c806a000 2023-07-06T01:20:17.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63c6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63c814f5b0 0x7f63c814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:17.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63c6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63c814f5b0 0x7f63c814fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47124/0 (socket says 172.21.15.22:47124) 2023-07-06T01:20:17.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63c6ffd700 1 -- 172.21.15.22:0/3604605390 learned_addr learned my addr 172.21.15.22:0/3604605390 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:17.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63c77fe700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63b0003f70 0x7f63c814d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:17.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63c7fff700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63c806a000 0x7f63c806a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:17.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63c6ffd700 1 -- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63c806a000 msgr2=0x7f63c806a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63c6ffd700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63c806a000 0x7f63c806a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63c6ffd700 1 -- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63b0003f70 msgr2=0x7f63c814d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:17.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63c6ffd700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63b0003f70 0x7f63c814d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:17.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63c6ffd700 1 -- 172.21.15.22:0/3604605390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c81508e0 con 0x7f63c814f5b0 2023-07-06T01:20:17.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63c6ffd700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63c814f5b0 0x7f63c814fa20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f63b000ba10 tx=0x7f63bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:17.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63affff700 1 -- 172.21.15.22:0/3604605390 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00e6c0 con 0x7f63c814f5b0 2023-07-06T01:20:17.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63affff700 1 -- 172.21.15.22:0/3604605390 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc0038a0 con 0x7f63c814f5b0 2023-07-06T01:20:17.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63cdb9b700 1 -- 172.21.15.22:0/3604605390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c8150bc0 con 0x7f63c814f5b0 2023-07-06T01:20:17.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63affff700 1 -- 172.21.15.22:0/3604605390 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc003c50 con 0x7f63c814f5b0 2023-07-06T01:20:17.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.787+0000 7f63cdb9b700 1 -- 172.21.15.22:0/3604605390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c814d640 con 0x7f63c814f5b0 2023-07-06T01:20:17.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.789+0000 7f63cdb9b700 1 -- 172.21.15.22:0/3604605390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c80b2cb0 con 0x7f63c814f5b0 2023-07-06T01:20:17.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.789+0000 7f63affff700 1 -- 172.21.15.22:0/3604605390 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f63bc013470 con 0x7f63c814f5b0 2023-07-06T01:20:17.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.789+0000 7f63affff700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f63b406ace0 0x7f63b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:17.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.790+0000 7f63affff700 1 -- 172.21.15.22:0/3604605390 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f63bc08aa40 con 0x7f63c814f5b0 2023-07-06T01:20:17.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.790+0000 7f63c77fe700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f63b406ace0 0x7f63b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:17.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.790+0000 7f63c77fe700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f63b406ace0 0x7f63b406d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f63b8007fd0 tx=0x7f63b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:17.798 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:17.794+0000 7f63affff700 1 -- 172.21.15.22:0/3604605390 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f63bc08ce60 con 0x7f63c814f5b0 2023-07-06T01:20:18.079 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.078+0000 7f63cdb9b700 1 -- 172.21.15.22:0/3604605390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f63c80744b0 con 0x7f63c814f5b0 2023-07-06T01:20:18.083 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.081+0000 7f63affff700 1 -- 172.21.15.22:0/3604605390 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f63bc055ff0 con 0x7f63c814f5b0 2023-07-06T01:20:18.085 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.083+0000 7f63cdb9b700 1 -- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f63b406ace0 msgr2=0x7f63b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:18.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.083+0000 7f63cdb9b700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f63b406ace0 0x7f63b406d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f63b8007fd0 tx=0x7f63b8009040 comp rx=0 tx=0).stop 2023-07-06T01:20:18.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.083+0000 7f63cdb9b700 1 -- 172.21.15.22:0/3604605390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63c814f5b0 msgr2=0x7f63c814fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:18.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.084+0000 7f63cdb9b700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63c814f5b0 0x7f63c814fa20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f63b000ba10 tx=0x7f63bc0047c0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.086 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.084+0000 7f63cdb9b700 1 -- 172.21.15.22:0/3604605390 shutdown_connections 2023-07-06T01:20:18.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.084+0000 7f63cdb9b700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63c806a000 0x7f63c806a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.084+0000 7f63cdb9b700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f63b406ace0 0x7f63b406d190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.084+0000 7f63cdb9b700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63b0003f70 0x7f63c814d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.084+0000 7f63cdb9b700 1 --2- 172.21.15.22:0/3604605390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63c814f5b0 0x7f63c814fa20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.084+0000 7f63cdb9b700 1 -- 172.21.15.22:0/3604605390 >> 172.21.15.22:0/3604605390 conn(0x7f63c806b110 msgr2=0x7f63c80b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:18.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.084+0000 7f63cdb9b700 1 -- 172.21.15.22:0/3604605390 shutdown_connections 2023-07-06T01:20:18.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.084+0000 7f63cdb9b700 1 -- 172.21.15.22:0/3604605390 wait complete. 2023-07-06T01:20:18.097 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-07-06T01:20:18.222 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.221+0000 7f795410c700 1 Processor -- start 2023-07-06T01:20:18.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.221+0000 7f795410c700 1 -- start start 2023-07-06T01:20:18.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.222+0000 7f795410c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f794c15a710 0x7f794c0ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:18.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.222+0000 7f795410c700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f794c0aecb0 0x7f794c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:18.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.222+0000 7f795410c700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794c0af270 con 0x7f794c0ceda0 2023-07-06T01:20:18.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.222+0000 7f795410c700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794c0cf220 con 0x7f794c15a710 2023-07-06T01:20:18.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.222+0000 7f795410c700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794c0af550 con 0x7f794c0aecb0 2023-07-06T01:20:18.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.222+0000 7f7951ea8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f794c15a710 0x7f794c0ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:18.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.222+0000 7f7951ea8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f794c15a710 0x7f794c0ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47130/0 (socket says 172.21.15.22:47130) 2023-07-06T01:20:18.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.222+0000 7f7951ea8700 1 -- 172.21.15.22:0/2862122220 learned_addr learned my addr 172.21.15.22:0/2862122220 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:18.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.222+0000 7f79516a7700 1 --2- 172.21.15.22:0/2862122220 >> v2:172.21.15.103:3300/0 conn(0x7f794c0aecb0 0x7f794c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:18.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.222+0000 7f7951ea8700 1 -- 172.21.15.22:0/2862122220 >> v2:172.21.15.103:3300/0 conn(0x7f794c0aecb0 msgr2=0x7f794c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:18.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.222+0000 7f7951ea8700 1 --2- 172.21.15.22:0/2862122220 >> v2:172.21.15.103:3300/0 conn(0x7f794c0aecb0 0x7f794c0ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.222+0000 7f7951ea8700 1 -- 172.21.15.22:0/2862122220 >> v1:172.21.15.103:6789/0 conn(0x7f794c0ceda0 legacy=0x7f794c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:18.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.222+0000 7f7951ea8700 1 -- 172.21.15.22:0/2862122220 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c0af830 con 0x7f794c15a710 2023-07-06T01:20:18.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.223+0000 7f7951ea8700 1 --2- 172.21.15.22:0/2862122220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c15a710 0x7f794c0ae770 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f794c071ac0 tx=0x7f7948009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c8929d6d03b459 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:18.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.223+0000 7f7950ea6700 1 -- 172.21.15.22:0/2862122220 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948004230 con 0x7f794c15a710 2023-07-06T01:20:18.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.223+0000 7f7950ea6700 1 -- 172.21.15.22:0/2862122220 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7948004390 con 0x7f794c15a710 2023-07-06T01:20:18.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.223+0000 7f7950ea6700 1 -- 172.21.15.22:0/2862122220 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79480036a0 con 0x7f794c15a710 2023-07-06T01:20:18.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.223+0000 7f795410c700 1 -- 172.21.15.22:0/2862122220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c15a710 msgr2=0x7f794c0ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:18.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.223+0000 7f795410c700 1 --2- 172.21.15.22:0/2862122220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c15a710 0x7f794c0ae770 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f794c071ac0 tx=0x7f7948009bd0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.223+0000 7f795410c700 1 -- 172.21.15.22:0/2862122220 shutdown_connections 2023-07-06T01:20:18.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.223+0000 7f795410c700 1 --2- 172.21.15.22:0/2862122220 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c15a710 0x7f794c0ae770 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.228 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.223+0000 7f795410c700 1 --2- 172.21.15.22:0/2862122220 >> v2:172.21.15.103:3300/0 conn(0x7f794c0aecb0 0x7f794c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.223+0000 7f795410c700 1 -- 172.21.15.22:0/2862122220 >> 172.21.15.22:0/2862122220 conn(0x7f794c0999f0 msgr2=0x7f794c099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:18.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.223+0000 7f795410c700 1 -- 172.21.15.22:0/2862122220 shutdown_connections 2023-07-06T01:20:18.229 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.223+0000 7f795410c700 1 -- 172.21.15.22:0/2862122220 wait complete. 2023-07-06T01:20:18.232 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.224+0000 7f795410c700 1 Processor -- start 2023-07-06T01:20:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.225+0000 7f795410c700 1 -- start start 2023-07-06T01:20:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.225+0000 7f795410c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c0aecb0 0x7f794c0b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:18.233 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.225+0000 7f795410c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f794c0ceda0 0x7f794c0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.225+0000 7f795410c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f794c15a710 0x7f794c0cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.225+0000 7f795410c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c1f56a0 con 0x7f794c0aecb0 2023-07-06T01:20:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.225+0000 7f795410c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794c0b6930 con 0x7f794c15a710 2023-07-06T01:20:18.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.225+0000 7f795410c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794c0c8a20 con 0x7f794c0ceda0 2023-07-06T01:20:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.225+0000 7f7951ea8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c0aecb0 0x7f794c0b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.225+0000 7f7951ea8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c0aecb0 0x7f794c0b63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47136/0 (socket says 172.21.15.22:47136) 2023-07-06T01:20:18.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.225+0000 7f7951ea8700 1 -- 172.21.15.22:0/4113347135 learned_addr learned my addr 172.21.15.22:0/4113347135 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.225+0000 7f79516a7700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f794c0ceda0 0x7f794c0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.226+0000 7f79526a9700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f794c15a710 0x7f794c0cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.226+0000 7f7951ea8700 1 -- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f794c0ceda0 msgr2=0x7f794c0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.226+0000 7f7951ea8700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f794c0ceda0 0x7f794c0cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.226+0000 7f7951ea8700 1 -- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f794c15a710 msgr2=0x7f794c0cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.226+0000 7f7951ea8700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f794c15a710 0x7f794c0cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.226+0000 7f7951ea8700 1 -- 172.21.15.22:0/4113347135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7948009550 con 0x7f794c0aecb0 2023-07-06T01:20:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.226+0000 7f79526a9700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f794c15a710 0x7f794c0cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.226+0000 7f7951ea8700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c0aecb0 0x7f794c0b63f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7948009130 tx=0x7f7948004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.226+0000 7f79427fc700 1 -- 172.21.15.22:0/4113347135 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948003dc0 con 0x7f794c0aecb0 2023-07-06T01:20:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.226+0000 7f79427fc700 1 -- 172.21.15.22:0/4113347135 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794801f070 con 0x7f794c0aecb0 2023-07-06T01:20:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.226+0000 7f795410c700 1 -- 172.21.15.22:0/4113347135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794c1f5940 con 0x7f794c0aecb0 2023-07-06T01:20:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.226+0000 7f79427fc700 1 -- 172.21.15.22:0/4113347135 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948011850 con 0x7f794c0aecb0 2023-07-06T01:20:18.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.226+0000 7f795410c700 1 -- 172.21.15.22:0/4113347135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794c1f5f00 con 0x7f794c0aecb0 2023-07-06T01:20:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.227+0000 7f795410c700 1 -- 172.21.15.22:0/4113347135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794c0ada80 con 0x7f794c0aecb0 2023-07-06T01:20:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.227+0000 7f79427fc700 1 -- 172.21.15.22:0/4113347135 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7948021030 con 0x7f794c0aecb0 2023-07-06T01:20:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.228+0000 7f79427fc700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f793006ace0 0x7f793006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.228+0000 7f79427fc700 1 -- 172.21.15.22:0/4113347135 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f794808a7a0 con 0x7f794c0aecb0 2023-07-06T01:20:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.228+0000 7f79516a7700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f793006ace0 0x7f793006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:18.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.228+0000 7f79516a7700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f793006ace0 0x7f793006d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f794c159630 tx=0x7f793c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:18.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.232+0000 7f79427fc700 1 -- 172.21.15.22:0/4113347135 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7948055c30 con 0x7f794c0aecb0 2023-07-06T01:20:18.514 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.512+0000 7f795410c700 1 -- 172.21.15.22:0/4113347135 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f794c067040 con 0x7f794c0aecb0 2023-07-06T01:20:18.517 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.515+0000 7f79427fc700 1 -- 172.21.15.22:0/4113347135 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7948059250 con 0x7f794c0aecb0 2023-07-06T01:20:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.517+0000 7f795410c700 1 -- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f793006ace0 msgr2=0x7f793006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.518+0000 7f795410c700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f793006ace0 0x7f793006d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f794c159630 tx=0x7f793c008040 comp rx=0 tx=0).stop 2023-07-06T01:20:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.518+0000 7f795410c700 1 -- 172.21.15.22:0/4113347135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c0aecb0 msgr2=0x7f794c0b63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.518+0000 7f795410c700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c0aecb0 0x7f794c0b63f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7948009130 tx=0x7f7948004750 comp rx=0 tx=0).stop 2023-07-06T01:20:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.518+0000 7f795410c700 1 -- 172.21.15.22:0/4113347135 shutdown_connections 2023-07-06T01:20:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.518+0000 7f795410c700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f793006ace0 0x7f793006d190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.518+0000 7f795410c700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f794c15a710 0x7f794c0cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.518+0000 7f795410c700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f794c0aecb0 0x7f794c0b63f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.518+0000 7f795410c700 1 --2- 172.21.15.22:0/4113347135 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f794c0ceda0 0x7f794c0cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.518+0000 7f795410c700 1 -- 172.21.15.22:0/4113347135 >> 172.21.15.22:0/4113347135 conn(0x7f794c0999f0 msgr2=0x7f794c159680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:18.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.518+0000 7f795410c700 1 -- 172.21.15.22:0/4113347135 shutdown_connections 2023-07-06T01:20:18.521 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.518+0000 7f795410c700 1 -- 172.21.15.22:0/4113347135 wait complete. 2023-07-06T01:20:18.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-07-06T01:20:18.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.657+0000 7f606e088700 1 Processor -- start 2023-07-06T01:20:18.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.657+0000 7f606e088700 1 -- start start 2023-07-06T01:20:18.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.657+0000 7f606e088700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60680c9a40 0x7f60680c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:18.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.657+0000 7f606e088700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f606815a690 0x7f60680ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:18.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.657+0000 7f606e088700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f60680cee60 0x7f606815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:18.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.657+0000 7f606e088700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60680ca350 con 0x7f60680cee60 2023-07-06T01:20:18.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.657+0000 7f606e088700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60680cd640 con 0x7f606815a690 2023-07-06T01:20:18.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.657+0000 7f606e088700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60680cd920 con 0x7f60680c9a40 2023-07-06T01:20:18.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f6067fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f60680cee60 0x7f606815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:18.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f6067fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f60680cee60 0x7f606815cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47148/0 (socket says 172.21.15.22:47148) 2023-07-06T01:20:18.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f6067fff700 1 -- 172.21.15.22:0/1242203167 learned_addr learned my addr 172.21.15.22:0/1242203167 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:18.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f60677fe700 1 --2- 172.21.15.22:0/1242203167 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60680c9a40 0x7f60680c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:18.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f6066ffd700 1 --2- 172.21.15.22:0/1242203167 >> v2:172.21.15.103:3300/0 conn(0x7f606815a690 0x7f60680ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:18.662 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f6067fff700 1 -- 172.21.15.22:0/1242203167 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60680c9a40 msgr2=0x7f60680c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:18.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f6067fff700 1 --2- 172.21.15.22:0/1242203167 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60680c9a40 0x7f60680c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f6067fff700 1 -- 172.21.15.22:0/1242203167 >> v2:172.21.15.103:3300/0 conn(0x7f606815a690 msgr2=0x7f60680ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:18.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f6067fff700 1 --2- 172.21.15.22:0/1242203167 >> v2:172.21.15.103:3300/0 conn(0x7f606815a690 0x7f60680ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f6067fff700 1 -- 172.21.15.22:0/1242203167 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60680cdc00 con 0x7f60680cee60 2023-07-06T01:20:18.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f6066ffd700 1 --2- 172.21.15.22:0/1242203167 >> v2:172.21.15.103:3300/0 conn(0x7f606815a690 0x7f60680ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:18.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f60677fe700 1 --2- 172.21.15.22:0/1242203167 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60680c9a40 0x7f60680c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:18.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f6067fff700 1 --2- 172.21.15.22:0/1242203167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60680cee60 0x7f606815cfd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6068068300 tx=0x7f606000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05a0579aad6176a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:18.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f60667fc700 1 -- 172.21.15.22:0/1242203167 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606000e070 con 0x7f60680cee60 2023-07-06T01:20:18.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f60667fc700 1 -- 172.21.15.22:0/1242203167 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606000b9c0 con 0x7f60680cee60 2023-07-06T01:20:18.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.658+0000 7f60667fc700 1 -- 172.21.15.22:0/1242203167 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606000bd70 con 0x7f60680cee60 2023-07-06T01:20:18.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.659+0000 7f606e088700 1 -- 172.21.15.22:0/1242203167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60680cee60 msgr2=0x7f606815cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:18.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.659+0000 7f606e088700 1 --2- 172.21.15.22:0/1242203167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60680cee60 0x7f606815cfd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6068068300 tx=0x7f606000b330 comp rx=0 tx=0).stop 2023-07-06T01:20:18.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.659+0000 7f606e088700 1 -- 172.21.15.22:0/1242203167 shutdown_connections 2023-07-06T01:20:18.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.659+0000 7f606e088700 1 --2- 172.21.15.22:0/1242203167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60680cee60 0x7f606815cfd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.659+0000 7f606e088700 1 --2- 172.21.15.22:0/1242203167 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60680c9a40 0x7f60680c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.659+0000 7f606e088700 1 --2- 172.21.15.22:0/1242203167 >> v2:172.21.15.103:3300/0 conn(0x7f606815a690 0x7f60680ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.659+0000 7f606e088700 1 -- 172.21.15.22:0/1242203167 >> 172.21.15.22:0/1242203167 conn(0x7f60680ccb00 msgr2=0x7f60680cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:18.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.659+0000 7f606e088700 1 -- 172.21.15.22:0/1242203167 shutdown_connections 2023-07-06T01:20:18.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.659+0000 7f606e088700 1 -- 172.21.15.22:0/1242203167 wait complete. 2023-07-06T01:20:18.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.660+0000 7f606e088700 1 Processor -- start 2023-07-06T01:20:18.667 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.660+0000 7f606e088700 1 -- start start 2023-07-06T01:20:18.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.661+0000 7f606e088700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60680c9a40 0x7f60680aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:18.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.661+0000 7f606e088700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60680cee60 0x7f60681552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:18.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.661+0000 7f606e088700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f606815a690 0x7f60681557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:18.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.661+0000 7f606e088700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606806c1f0 con 0x7f606815a690 2023-07-06T01:20:18.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.661+0000 7f606e088700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606806c4d0 con 0x7f60680c9a40 2023-07-06T01:20:18.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.661+0000 7f606e088700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606806c7b0 con 0x7f60680cee60 2023-07-06T01:20:18.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.661+0000 7f6067fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f606815a690 0x7f60681557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:18.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.661+0000 7f6067fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f606815a690 0x7f60681557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47154/0 (socket says 172.21.15.22:47154) 2023-07-06T01:20:18.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.661+0000 7f6067fff700 1 -- 172.21.15.22:0/3989738635 learned_addr learned my addr 172.21.15.22:0/3989738635 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:18.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.661+0000 7f60677fe700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60680c9a40 0x7f60680aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:18.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.661+0000 7f6066ffd700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60680cee60 0x7f60681552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:18.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.662+0000 7f6067fff700 1 -- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60680cee60 msgr2=0x7f60681552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:18.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.662+0000 7f6067fff700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60680cee60 0x7f60681552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.662+0000 7f6067fff700 1 -- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60680c9a40 msgr2=0x7f60680aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.662+0000 7f6067fff700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60680c9a40 0x7f60680aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.662+0000 7f6067fff700 1 -- 172.21.15.22:0/3989738635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6060009b30 con 0x7f606815a690 2023-07-06T01:20:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.662+0000 7f6067fff700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f606815a690 0x7f60681557e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f606000b370 tx=0x7f6060004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.662+0000 7f605ffff700 1 -- 172.21.15.22:0/3989738635 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060004250 con 0x7f606815a690 2023-07-06T01:20:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.662+0000 7f605ffff700 1 -- 172.21.15.22:0/3989738635 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60600043f0 con 0x7f606815a690 2023-07-06T01:20:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.662+0000 7f605ffff700 1 -- 172.21.15.22:0/3989738635 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60600047a0 con 0x7f606815a690 2023-07-06T01:20:18.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.662+0000 7f606e088700 1 -- 172.21.15.22:0/3989738635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606806ca90 con 0x7f606815a690 2023-07-06T01:20:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.662+0000 7f606e088700 1 -- 172.21.15.22:0/3989738635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60681f1ad0 con 0x7f606815a690 2023-07-06T01:20:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.663+0000 7f606e088700 1 -- 172.21.15.22:0/3989738635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f606806cfb0 con 0x7f606815a690 2023-07-06T01:20:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.663+0000 7f605ffff700 1 -- 172.21.15.22:0/3989738635 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f606001b430 con 0x7f606815a690 2023-07-06T01:20:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.664+0000 7f605ffff700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f604806ace0 0x7f604806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.664+0000 7f605ffff700 1 -- 172.21.15.22:0/3989738635 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f606008a240 con 0x7f606815a690 2023-07-06T01:20:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.664+0000 7f60677fe700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f604806ace0 0x7f604806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:18.671 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.664+0000 7f60677fe700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f604806ace0 0x7f604806d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f605800ab70 tx=0x7f6058008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:18.672 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.668+0000 7f605ffff700 1 -- 172.21.15.22:0/3989738635 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f606806cfb0 con 0x7f606815a690 2023-07-06T01:20:18.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.945+0000 7f606e088700 1 -- 172.21.15.22:0/3989738635 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f606806cfb0 con 0x7f606815a690 2023-07-06T01:20:18.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.948+0000 7f605ffff700 1 -- 172.21.15.22:0/3989738635 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f606806cfb0 con 0x7f606815a690 2023-07-06T01:20:18.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.950+0000 7f606e088700 1 -- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f604806ace0 msgr2=0x7f604806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:18.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.950+0000 7f606e088700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f604806ace0 0x7f604806d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f605800ab70 tx=0x7f6058008040 comp rx=0 tx=0).stop 2023-07-06T01:20:18.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.950+0000 7f606e088700 1 -- 172.21.15.22:0/3989738635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f606815a690 msgr2=0x7f60681557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:18.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.950+0000 7f606e088700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f606815a690 0x7f60681557e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f606000b370 tx=0x7f6060004ad0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.950+0000 7f606e088700 1 -- 172.21.15.22:0/3989738635 shutdown_connections 2023-07-06T01:20:18.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.950+0000 7f606e088700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f606815a690 0x7f60681557e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.950+0000 7f606e088700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f604806ace0 0x7f604806d190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.950+0000 7f606e088700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60680c9a40 0x7f60680aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.950+0000 7f606e088700 1 --2- 172.21.15.22:0/3989738635 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60680cee60 0x7f60681552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:18.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.950+0000 7f606e088700 1 -- 172.21.15.22:0/3989738635 >> 172.21.15.22:0/3989738635 conn(0x7f60680ccb00 msgr2=0x7f6068158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.950+0000 7f606e088700 1 -- 172.21.15.22:0/3989738635 shutdown_connections 2023-07-06T01:20:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:18.950+0000 7f606e088700 1 -- 172.21.15.22:0/3989738635 wait complete. 2023-07-06T01:20:18.964 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-07-06T01:20:19.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.088+0000 7ff4b5957700 1 Processor -- start 2023-07-06T01:20:19.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.088+0000 7ff4b5957700 1 -- start start 2023-07-06T01:20:19.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4b5957700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff4b00b3b70 0x7ff4b00d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4b5957700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b00cebd0 con 0x7ff4b00b36f0 2023-07-06T01:20:19.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4b5957700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b00cdfc0 con 0x7ff4b00b40c0 2023-07-06T01:20:19.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4b5957700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b015cff0 con 0x7ff4b00b3b70 2023-07-06T01:20:19.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4aeffd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff4b00b36f0 0x7ff4b00cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37902/0 (socket says 172.21.15.22:37902) 2023-07-06T01:20:19.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4aeffd700 1 -- 172.21.15.22:0/1715336888 learned_addr learned my addr 172.21.15.22:0/1715336888 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:19.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271996703 0 0) 0x7ff4b00cebd0 con 0x7ff4b00b36f0 2023-07-06T01:20:19.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4980035b0 con 0x7ff4b00b36f0 2023-07-06T01:20:19.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4ae7fc700 1 --2- 172.21.15.22:0/1715336888 >> v2:172.21.15.103:3300/0 conn(0x7ff4b00b3b70 0x7ff4b00d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4028467454 0 0) 0x7ff4980035b0 con 0x7ff4b00b36f0 2023-07-06T01:20:19.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 >> v2:172.21.15.103:3300/0 conn(0x7ff4b00b3b70 msgr2=0x7ff4b00d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:19.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4adffb700 1 --2- 172.21.15.22:0/1715336888 >> v2:172.21.15.103:3300/0 conn(0x7ff4b00b3b70 0x7ff4b00d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 >> v1:172.21.15.103:6789/0 conn(0x7ff4b00b40c0 legacy=0x7ff4b00b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b00ce2a0 con 0x7ff4b00b36f0 2023-07-06T01:20:19.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.089+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7ff4a4001670 con 0x7ff4b00b36f0 2023-07-06T01:20:19.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 >> v1:172.21.15.22:6789/0 conn(0x7ff4b00b36f0 legacy=0x7ff4b00cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4adffb700 1 --2- 172.21.15.22:0/1715336888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff498003a00 0x7ff498003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4adffb700 1 --2- 172.21.15.22:0/1715336888 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff498004850 0x7ff498004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4adffb700 1 --2- 172.21.15.22:0/1715336888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498005320 0x7ff49800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b015cff0 con 0x7ff498005320 2023-07-06T01:20:19.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49800c250 con 0x7ff498003a00 2023-07-06T01:20:19.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49800c410 con 0x7ff498004850 2023-07-06T01:20:19.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4af7fe700 1 --2- 172.21.15.22:0/1715336888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498005320 0x7ff49800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4aeffd700 1 --2- 172.21.15.22:0/1715336888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff498003a00 0x7ff498003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4ae7fc700 1 --2- 172.21.15.22:0/1715336888 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff498004850 0x7ff498004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4af7fe700 1 -- 172.21.15.22:0/1715336888 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff498004850 msgr2=0x7ff498004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:19.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4af7fe700 1 --2- 172.21.15.22:0/1715336888 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff498004850 0x7ff498004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4af7fe700 1 -- 172.21.15.22:0/1715336888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff498003a00 msgr2=0x7ff498003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:19.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4af7fe700 1 --2- 172.21.15.22:0/1715336888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff498003a00 0x7ff498003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4af7fe700 1 -- 172.21.15.22:0/1715336888 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980035b0 con 0x7ff498005320 2023-07-06T01:20:19.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4af7fe700 1 -- 172.21.15.22:0/1715336888 reap_dead start 2023-07-06T01:20:19.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4af7fe700 1 --2- 172.21.15.22:0/1715336888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498005320 0x7ff49800ba10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800c190 tx=0x7ff498005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd2c711eecf26de server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:19.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8001db0 con 0x7ff498005320 2023-07-06T01:20:19.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a800ccd0 con 0x7ff498005320 2023-07-06T01:20:19.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.090+0000 7ff4adffb700 1 -- 172.21.15.22:0/1715336888 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8003c40 con 0x7ff498005320 2023-07-06T01:20:19.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.091+0000 7ff4b5957700 1 -- 172.21.15.22:0/1715336888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498005320 msgr2=0x7ff49800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.091+0000 7ff4b5957700 1 --2- 172.21.15.22:0/1715336888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498005320 0x7ff49800ba10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800c190 tx=0x7ff498005880 comp rx=0 tx=0).stop 2023-07-06T01:20:19.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.091+0000 7ff4b5957700 1 -- 172.21.15.22:0/1715336888 shutdown_connections 2023-07-06T01:20:19.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.091+0000 7ff4b5957700 1 --2- 172.21.15.22:0/1715336888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498005320 0x7ff49800ba10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.091+0000 7ff4b5957700 1 -- 172.21.15.22:0/1715336888 >> 172.21.15.22:0/1715336888 conn(0x7ff4b009c2e0 msgr2=0x7ff4b00d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:19.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.091+0000 7ff4b5957700 1 -- 172.21.15.22:0/1715336888 shutdown_connections 2023-07-06T01:20:19.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.091+0000 7ff4b5957700 1 -- 172.21.15.22:0/1715336888 wait complete. 2023-07-06T01:20:19.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.092+0000 7ff4b5957700 1 Processor -- start 2023-07-06T01:20:19.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.092+0000 7ff4b5957700 1 -- start start 2023-07-06T01:20:19.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4b5957700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498005320 0x7ff4b015d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4b5957700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4b015d670 0x7ff4b015eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4b5957700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b015f2e0 0x7ff4b015f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4b5957700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b01f1ef0 con 0x7ff498005320 2023-07-06T01:20:19.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4b5957700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00c9300 con 0x7ff4b015f2e0 2023-07-06T01:20:19.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4b5957700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b015fc90 con 0x7ff4b015d670 2023-07-06T01:20:19.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4aeffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498005320 0x7ff4b015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4aeffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498005320 0x7ff4b015d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47170/0 (socket says 172.21.15.22:47170) 2023-07-06T01:20:19.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4aeffd700 1 -- 172.21.15.22:0/755465667 learned_addr learned my addr 172.21.15.22:0/755465667 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:19.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4af7fe700 1 --2- 172.21.15.22:0/755465667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b015f2e0 0x7ff4b015f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4aeffd700 1 -- 172.21.15.22:0/755465667 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4b015d670 msgr2=0x7ff4b015eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4aeffd700 1 --2- 172.21.15.22:0/755465667 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4b015d670 0x7ff4b015eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4aeffd700 1 -- 172.21.15.22:0/755465667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b015f2e0 msgr2=0x7ff4b015f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4aeffd700 1 --2- 172.21.15.22:0/755465667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b015f2e0 0x7ff4b015f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.093+0000 7ff4aeffd700 1 -- 172.21.15.22:0/755465667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b00c8780 con 0x7ff498005320 2023-07-06T01:20:19.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.094+0000 7ff4aeffd700 1 --2- 172.21.15.22:0/755465667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498005320 0x7ff4b015d130 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40072c0 tx=0x7ff4a4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:19.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.094+0000 7ff497fff700 1 -- 172.21.15.22:0/755465667 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40077a0 con 0x7ff498005320 2023-07-06T01:20:19.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.094+0000 7ff497fff700 1 -- 172.21.15.22:0/755465667 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a400f8d0 con 0x7ff498005320 2023-07-06T01:20:19.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.094+0000 7ff4b5957700 1 -- 172.21.15.22:0/755465667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b00c8a60 con 0x7ff498005320 2023-07-06T01:20:19.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.094+0000 7ff497fff700 1 -- 172.21.15.22:0/755465667 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400fc80 con 0x7ff498005320 2023-07-06T01:20:19.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.094+0000 7ff4b5957700 1 -- 172.21.15.22:0/755465667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b00c9020 con 0x7ff498005320 2023-07-06T01:20:19.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.095+0000 7ff4b5957700 1 -- 172.21.15.22:0/755465667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b00b4840 con 0x7ff498005320 2023-07-06T01:20:19.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.095+0000 7ff497fff700 1 -- 172.21.15.22:0/755465667 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff4a401d460 con 0x7ff498005320 2023-07-06T01:20:19.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.096+0000 7ff497fff700 1 --2- 172.21.15.22:0/755465667 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff49c06ace0 0x7ff49c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.096+0000 7ff497fff700 1 -- 172.21.15.22:0/755465667 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff4a4089050 con 0x7ff498005320 2023-07-06T01:20:19.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.096+0000 7ff4ae7fc700 1 --2- 172.21.15.22:0/755465667 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff49c06ace0 0x7ff49c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.097+0000 7ff4ae7fc700 1 --2- 172.21.15.22:0/755465667 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff49c06ace0 0x7ff49c06d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0155b70 tx=0x7ff4a00093c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:19.105 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.100+0000 7ff497fff700 1 -- 172.21.15.22:0/755465667 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff4a40544e0 con 0x7ff498005320 2023-07-06T01:20:19.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.378+0000 7ff4b5957700 1 -- 172.21.15.22:0/755465667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7ff4b00744b0 con 0x7ff498005320 2023-07-06T01:20:19.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.381+0000 7ff497fff700 1 -- 172.21.15.22:0/755465667 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff4a4057b00 con 0x7ff498005320 2023-07-06T01:20:19.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.383+0000 7ff4b5957700 1 -- 172.21.15.22:0/755465667 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff49c06ace0 msgr2=0x7ff49c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.383+0000 7ff4b5957700 1 --2- 172.21.15.22:0/755465667 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff49c06ace0 0x7ff49c06d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0155b70 tx=0x7ff4a00093c0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.383+0000 7ff4b5957700 1 -- 172.21.15.22:0/755465667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498005320 msgr2=0x7ff4b015d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.385 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.383+0000 7ff4b5957700 1 --2- 172.21.15.22:0/755465667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498005320 0x7ff4b015d130 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40072c0 tx=0x7ff4a4007680 comp rx=0 tx=0).stop 2023-07-06T01:20:19.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.383+0000 7ff4b5957700 1 -- 172.21.15.22:0/755465667 shutdown_connections 2023-07-06T01:20:19.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.383+0000 7ff4b5957700 1 --2- 172.21.15.22:0/755465667 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff49c06ace0 0x7ff49c06d190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.383+0000 7ff4b5957700 1 --2- 172.21.15.22:0/755465667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4b015f2e0 0x7ff4b015f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.383+0000 7ff4b5957700 1 --2- 172.21.15.22:0/755465667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff498005320 0x7ff4b015d130 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.383+0000 7ff4b5957700 1 --2- 172.21.15.22:0/755465667 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4b015d670 0x7ff4b015eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.383+0000 7ff4b5957700 1 -- 172.21.15.22:0/755465667 >> 172.21.15.22:0/755465667 conn(0x7ff4b009c2e0 msgr2=0x7ff4b00d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:19.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.383+0000 7ff4b5957700 1 -- 172.21.15.22:0/755465667 shutdown_connections 2023-07-06T01:20:19.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.383+0000 7ff4b5957700 1 -- 172.21.15.22:0/755465667 wait complete. 2023-07-06T01:20:19.397 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-07-06T01:20:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.521+0000 7f485e97d700 1 Processor -- start 2023-07-06T01:20:19.523 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.521+0000 7f485e97d700 1 -- start start 2023-07-06T01:20:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.522+0000 7f485e97d700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f485815cb50 0x7f485806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.522+0000 7f485e97d700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f485806a540 0x7f48580ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.522+0000 7f485e97d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f48580cf090 0x7f485806cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.525 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.522+0000 7f485e97d700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48580af440 con 0x7f485815cb50 2023-07-06T01:20:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.522+0000 7f485e97d700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48580ca2b0 con 0x7f485806a540 2023-07-06T01:20:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.522+0000 7f485e97d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485806a970 con 0x7f48580cf090 2023-07-06T01:20:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.522+0000 7f4857fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f485815cb50 0x7f485806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.522+0000 7f4857fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f485815cb50 0x7f485806a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47178/0 (socket says 172.21.15.22:47178) 2023-07-06T01:20:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.522+0000 7f4857fff700 1 -- 172.21.15.22:0/646804737 learned_addr learned my addr 172.21.15.22:0/646804737 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.522+0000 7f485cf1a700 1 --2- 172.21.15.22:0/646804737 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f48580cf090 0x7f485806cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.526 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.522+0000 7f48577fe700 1 --2- 172.21.15.22:0/646804737 >> v2:172.21.15.103:3300/0 conn(0x7f485806a540 0x7f48580ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.523+0000 7f4857fff700 1 -- 172.21.15.22:0/646804737 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f48580cf090 msgr2=0x7f485806cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:19.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.523+0000 7f4857fff700 1 --2- 172.21.15.22:0/646804737 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f48580cf090 0x7f485806cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.523+0000 7f4857fff700 1 -- 172.21.15.22:0/646804737 >> v2:172.21.15.103:3300/0 conn(0x7f485806a540 msgr2=0x7f48580ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:19.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.523+0000 7f4857fff700 1 --2- 172.21.15.22:0/646804737 >> v2:172.21.15.103:3300/0 conn(0x7f485806a540 0x7f48580ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.523+0000 7f4857fff700 1 -- 172.21.15.22:0/646804737 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48580cfc90 con 0x7f485815cb50 2023-07-06T01:20:19.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.523+0000 7f4857fff700 1 --2- 172.21.15.22:0/646804737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f485815cb50 0x7f485806a000 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f48580d0d00 tx=0x7f4848009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba47736068b6534 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:19.527 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.523+0000 7f4856ffd700 1 -- 172.21.15.22:0/646804737 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848011830 con 0x7f485815cb50 2023-07-06T01:20:19.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.523+0000 7f4856ffd700 1 -- 172.21.15.22:0/646804737 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4848011990 con 0x7f485815cb50 2023-07-06T01:20:19.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.523+0000 7f4856ffd700 1 -- 172.21.15.22:0/646804737 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848011d20 con 0x7f485815cb50 2023-07-06T01:20:19.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.523+0000 7f485e97d700 1 -- 172.21.15.22:0/646804737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f485815cb50 msgr2=0x7f485806a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.523+0000 7f485e97d700 1 --2- 172.21.15.22:0/646804737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f485815cb50 0x7f485806a000 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f48580d0d00 tx=0x7f4848009bd0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.524+0000 7f485e97d700 1 -- 172.21.15.22:0/646804737 shutdown_connections 2023-07-06T01:20:19.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.524+0000 7f485e97d700 1 --2- 172.21.15.22:0/646804737 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f48580cf090 0x7f485806cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.524+0000 7f485e97d700 1 --2- 172.21.15.22:0/646804737 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f485815cb50 0x7f485806a000 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.528 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.524+0000 7f485e97d700 1 --2- 172.21.15.22:0/646804737 >> v2:172.21.15.103:3300/0 conn(0x7f485806a540 0x7f48580ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.524+0000 7f485e97d700 1 -- 172.21.15.22:0/646804737 >> 172.21.15.22:0/646804737 conn(0x7f485806b110 msgr2=0x7f485806b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:19.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.524+0000 7f485e97d700 1 -- 172.21.15.22:0/646804737 shutdown_connections 2023-07-06T01:20:19.529 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.524+0000 7f485e97d700 1 -- 172.21.15.22:0/646804737 wait complete. 2023-07-06T01:20:19.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.525+0000 7f485e97d700 1 Processor -- start 2023-07-06T01:20:19.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.525+0000 7f485e97d700 1 -- start start 2023-07-06T01:20:19.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.526+0000 7f485e97d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f485806a540 0x7f48580c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.526+0000 7f485e97d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48580cf090 0x7f485815daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.526+0000 7f485e97d700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f485815cb50 0x7f485815e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.531 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.526+0000 7f485e97d700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48580c4ef0 con 0x7f48580cf090 2023-07-06T01:20:19.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.526+0000 7f485e97d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48580ad830 con 0x7f485806a540 2023-07-06T01:20:19.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.526+0000 7f485e97d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48580adb10 con 0x7f485815cb50 2023-07-06T01:20:19.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.526+0000 7f48577fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48580cf090 0x7f485815daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.526+0000 7f48577fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48580cf090 0x7f485815daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47186/0 (socket says 172.21.15.22:47186) 2023-07-06T01:20:19.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.526+0000 7f48577fe700 1 -- 172.21.15.22:0/2548951605 learned_addr learned my addr 172.21.15.22:0/2548951605 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:19.532 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f485cf1a700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f485815cb50 0x7f485815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f4857fff700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f485806a540 0x7f48580c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f48577fe700 1 -- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f485815cb50 msgr2=0x7f485815e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f48577fe700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f485815cb50 0x7f485815e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f48577fe700 1 -- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f485806a540 msgr2=0x7f48580c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f48577fe700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f485806a540 0x7f48580c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f48577fe700 1 -- 172.21.15.22:0/2548951605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4848009550 con 0x7f48580cf090 2023-07-06T01:20:19.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f485cf1a700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f485815cb50 0x7f485815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:19.533 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f4857fff700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f485806a540 0x7f48580c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:19.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f48577fe700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48580cf090 0x7f485815daf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f484c00e870 tx=0x7f484c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:19.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f4854ff9700 1 -- 172.21.15.22:0/2548951605 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c00c640 con 0x7f48580cf090 2023-07-06T01:20:19.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f4854ff9700 1 -- 172.21.15.22:0/2548951605 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484c00c7a0 con 0x7f48580cf090 2023-07-06T01:20:19.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f485e97d700 1 -- 172.21.15.22:0/2548951605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48580addf0 con 0x7f48580cf090 2023-07-06T01:20:19.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f4854ff9700 1 -- 172.21.15.22:0/2548951605 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c00cb30 con 0x7f48580cf090 2023-07-06T01:20:19.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.527+0000 7f485e97d700 1 -- 172.21.15.22:0/2548951605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48580ae3b0 con 0x7f48580cf090 2023-07-06T01:20:19.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.528+0000 7f485e97d700 1 -- 172.21.15.22:0/2548951605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48580c3600 con 0x7f48580cf090 2023-07-06T01:20:19.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.530+0000 7f4854ff9700 1 -- 172.21.15.22:0/2548951605 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f484c00cc90 con 0x7f48580cf090 2023-07-06T01:20:19.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.530+0000 7f4854ff9700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f484406ad10 0x7f484406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.530+0000 7f4854ff9700 1 -- 172.21.15.22:0/2548951605 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f484c0897a0 con 0x7f48580cf090 2023-07-06T01:20:19.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.530+0000 7f4857fff700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f484406ad10 0x7f484406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.531+0000 7f4857fff700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f484406ad10 0x7f484406d1c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4848009fd0 tx=0x7f4848008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:19.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.534+0000 7f4854ff9700 1 -- 172.21.15.22:0/2548951605 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f484c054c80 con 0x7f48580cf090 2023-07-06T01:20:19.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.815+0000 7f485e97d700 1 -- 172.21.15.22:0/2548951605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f48580ae5d0 con 0x7f48580cf090 2023-07-06T01:20:19.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.818+0000 7f4854ff9700 1 -- 172.21.15.22:0/2548951605 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f484c0582a0 con 0x7f48580cf090 2023-07-06T01:20:19.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.820+0000 7f485e97d700 1 -- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f484406ad10 msgr2=0x7f484406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.820+0000 7f485e97d700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f484406ad10 0x7f484406d1c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4848009fd0 tx=0x7f4848008030 comp rx=0 tx=0).stop 2023-07-06T01:20:19.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.820+0000 7f485e97d700 1 -- 172.21.15.22:0/2548951605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48580cf090 msgr2=0x7f485815daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.820+0000 7f485e97d700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48580cf090 0x7f485815daf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f484c00e870 tx=0x7f484c00ec30 comp rx=0 tx=0).stop 2023-07-06T01:20:19.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.820+0000 7f485e97d700 1 -- 172.21.15.22:0/2548951605 shutdown_connections 2023-07-06T01:20:19.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.820+0000 7f485e97d700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f485815cb50 0x7f485815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.820+0000 7f485e97d700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f484406ad10 0x7f484406d1c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.820+0000 7f485e97d700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f485806a540 0x7f48580c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.820+0000 7f485e97d700 1 --2- 172.21.15.22:0/2548951605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f48580cf090 0x7f485815daf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.820+0000 7f485e97d700 1 -- 172.21.15.22:0/2548951605 >> 172.21.15.22:0/2548951605 conn(0x7f485806b110 msgr2=0x7f485806cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:19.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.820+0000 7f485e97d700 1 -- 172.21.15.22:0/2548951605 shutdown_connections 2023-07-06T01:20:19.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.820+0000 7f485e97d700 1 -- 172.21.15.22:0/2548951605 wait complete. 2023-07-06T01:20:19.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-07-06T01:20:19.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.958+0000 7f687cb48700 1 Processor -- start 2023-07-06T01:20:19.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.958+0000 7f687cb48700 1 -- start start 2023-07-06T01:20:19.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.959+0000 7f687cb48700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f68780cc320 0x7f687806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.959+0000 7f687cb48700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68780af670 con 0x7f68780af2a0 2023-07-06T01:20:19.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.959+0000 7f687cb48700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687806d030 con 0x7f687806a710 2023-07-06T01:20:19.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.959+0000 7f687cb48700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68780cc780 con 0x7f68780cc320 2023-07-06T01:20:19.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.959+0000 7f687659c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f68780af2a0 0x7f687806a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37936/0 (socket says 172.21.15.22:37936) 2023-07-06T01:20:19.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.959+0000 7f687659c700 1 -- 172.21.15.22:0/4254562197 learned_addr learned my addr 172.21.15.22:0/4254562197 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:19.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.959+0000 7f6876d9d700 1 --2- 172.21.15.22:0/4254562197 >> v2:172.21.15.103:3300/0 conn(0x7f68780cc320 0x7f687806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.959+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780019977 0 0) 0x7f68780af670 con 0x7f68780af2a0 2023-07-06T01:20:19.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.959+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68600035b0 con 0x7f68780af2a0 2023-07-06T01:20:19.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1420007415 0 0) 0x7f68600035b0 con 0x7f68780af2a0 2023-07-06T01:20:19.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 >> v2:172.21.15.103:3300/0 conn(0x7f68780cc320 msgr2=0x7f687806caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:19.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 --2- 172.21.15.22:0/4254562197 >> v2:172.21.15.103:3300/0 conn(0x7f68780cc320 0x7f687806caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 >> v1:172.21.15.103:6789/0 conn(0x7f687806a710 legacy=0x7f68780cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68780cd560 con 0x7f68780af2a0 2023-07-06T01:20:19.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f686c001670 con 0x7f68780af2a0 2023-07-06T01:20:19.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 >> v1:172.21.15.22:6789/0 conn(0x7f68780af2a0 legacy=0x7f687806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 --2- 172.21.15.22:0/4254562197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6860003a00 0x7f6860003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 --2- 172.21.15.22:0/4254562197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6860004850 0x7f6860004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 --2- 172.21.15.22:0/4254562197 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6860005200 0x7f686000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68780cc780 con 0x7f6860003a00 2023-07-06T01:20:19.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686000c080 con 0x7f6860004850 2023-07-06T01:20:19.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686000c240 con 0x7f6860005200 2023-07-06T01:20:19.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687659c700 1 --2- 172.21.15.22:0/4254562197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6860003a00 0x7f6860003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f6875d9b700 1 --2- 172.21.15.22:0/4254562197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6860004850 0x7f6860004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f6876d9d700 1 --2- 172.21.15.22:0/4254562197 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6860005200 0x7f686000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687659c700 1 -- 172.21.15.22:0/4254562197 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6860005200 msgr2=0x7f686000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:19.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687659c700 1 --2- 172.21.15.22:0/4254562197 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6860005200 0x7f686000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687659c700 1 -- 172.21.15.22:0/4254562197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6860004850 msgr2=0x7f6860004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:19.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687659c700 1 --2- 172.21.15.22:0/4254562197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6860004850 0x7f6860004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f6876d9d700 1 -- 172.21.15.22:0/4254562197 reap_dead start 2023-07-06T01:20:19.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687659c700 1 -- 172.21.15.22:0/4254562197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68600035b0 con 0x7f6860003a00 2023-07-06T01:20:19.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687659c700 1 --2- 172.21.15.22:0/4254562197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6860003a00 0x7f6860003e10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f686c00c240 tx=0x7f6860005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6345765b3dde1b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:19.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c007c10 con 0x7f6860003a00 2023-07-06T01:20:19.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686c00cd60 con 0x7f6860003a00 2023-07-06T01:20:19.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.960+0000 7f687559a700 1 -- 172.21.15.22:0/4254562197 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c005b80 con 0x7f6860003a00 2023-07-06T01:20:19.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.961+0000 7f687cb48700 1 -- 172.21.15.22:0/4254562197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6860003a00 msgr2=0x7f6860003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.961+0000 7f687cb48700 1 --2- 172.21.15.22:0/4254562197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6860003a00 0x7f6860003e10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f686c00c240 tx=0x7f6860005880 comp rx=0 tx=0).stop 2023-07-06T01:20:19.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.961+0000 7f687cb48700 1 -- 172.21.15.22:0/4254562197 shutdown_connections 2023-07-06T01:20:19.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.961+0000 7f687cb48700 1 --2- 172.21.15.22:0/4254562197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6860003a00 0x7f6860003e10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.961+0000 7f687cb48700 1 -- 172.21.15.22:0/4254562197 >> 172.21.15.22:0/4254562197 conn(0x7f68780b1890 msgr2=0x7f68780b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:19.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.961+0000 7f687cb48700 1 -- 172.21.15.22:0/4254562197 shutdown_connections 2023-07-06T01:20:19.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.961+0000 7f687cb48700 1 -- 172.21.15.22:0/4254562197 wait complete. 2023-07-06T01:20:19.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.963+0000 7f687cb48700 1 Processor -- start 2023-07-06T01:20:19.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.963+0000 7f687cb48700 1 -- start start 2023-07-06T01:20:19.971 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f687cb48700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6860003a00 0x7f68780b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f687cb48700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68780adb60 0x7f687815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f687cb48700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68780adfd0 0x7f687815f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f687cb48700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68780b5270 con 0x7f68780adfd0 2023-07-06T01:20:19.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f687cb48700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68780b45f0 con 0x7f68780adb60 2023-07-06T01:20:19.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f687cb48700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68780b48d0 con 0x7f6860003a00 2023-07-06T01:20:19.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f6876d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68780adfd0 0x7f687815f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f6876d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68780adfd0 0x7f687815f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47208/0 (socket says 172.21.15.22:47208) 2023-07-06T01:20:19.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f6876d9d700 1 -- 172.21.15.22:0/1375618211 learned_addr learned my addr 172.21.15.22:0/1375618211 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:19.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f687659c700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6860003a00 0x7f68780b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f6875d9b700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68780adb60 0x7f687815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f6876d9d700 1 -- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6860003a00 msgr2=0x7f68780b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f6876d9d700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6860003a00 0x7f68780b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f6876d9d700 1 -- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68780adb60 msgr2=0x7f687815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:19.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f6876d9d700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68780adb60 0x7f687815ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:19.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f6876d9d700 1 -- 172.21.15.22:0/1375618211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68780b4bb0 con 0x7f68780adfd0 2023-07-06T01:20:19.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f6875d9b700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68780adb60 0x7f687815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:19.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.964+0000 7f6876d9d700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68780adfd0 0x7f687815f220 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f68780cb2f0 tx=0x7f6870011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:19.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.965+0000 7f6866ffd700 1 -- 172.21.15.22:0/1375618211 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687000e040 con 0x7f68780adfd0 2023-07-06T01:20:19.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.965+0000 7f6866ffd700 1 -- 172.21.15.22:0/1375618211 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687000f040 con 0x7f68780adfd0 2023-07-06T01:20:19.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.965+0000 7f687cb48700 1 -- 172.21.15.22:0/1375618211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68780b4e90 con 0x7f68780adfd0 2023-07-06T01:20:19.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.965+0000 7f6866ffd700 1 -- 172.21.15.22:0/1375618211 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687000fd00 con 0x7f68780adfd0 2023-07-06T01:20:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.965+0000 7f687cb48700 1 -- 172.21.15.22:0/1375618211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687815f940 con 0x7f68780adfd0 2023-07-06T01:20:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.966+0000 7f687cb48700 1 -- 172.21.15.22:0/1375618211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68781f3740 con 0x7f68780adfd0 2023-07-06T01:20:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.966+0000 7f6866ffd700 1 -- 172.21.15.22:0/1375618211 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6870015430 con 0x7f68780adfd0 2023-07-06T01:20:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.966+0000 7f6866ffd700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f685806ace0 0x7f685806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.967+0000 7f6866ffd700 1 -- 172.21.15.22:0/1375618211 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f68700898b0 con 0x7f68780adfd0 2023-07-06T01:20:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.967+0000 7f687659c700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f685806ace0 0x7f685806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:19.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.967+0000 7f687659c700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f685806ace0 0x7f685806d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f686c000c00 tx=0x7f686c006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:19.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:19.971+0000 7f6866ffd700 1 -- 172.21.15.22:0/1375618211 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f68781f3740 con 0x7f68780adfd0 2023-07-06T01:20:20.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.249+0000 7f687cb48700 1 -- 172.21.15.22:0/1375618211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f68781f3740 con 0x7f68780adfd0 2023-07-06T01:20:20.253 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.252+0000 7f6866ffd700 1 -- 172.21.15.22:0/1375618211 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f68781f3740 con 0x7f68780adfd0 2023-07-06T01:20:20.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.254+0000 7f687cb48700 1 -- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f685806ace0 msgr2=0x7f685806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:20.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.254+0000 7f687cb48700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f685806ace0 0x7f685806d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f686c000c00 tx=0x7f686c006040 comp rx=0 tx=0).stop 2023-07-06T01:20:20.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.254+0000 7f687cb48700 1 -- 172.21.15.22:0/1375618211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68780adfd0 msgr2=0x7f687815f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:20.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.254+0000 7f687cb48700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68780adfd0 0x7f687815f220 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f68780cb2f0 tx=0x7f6870011ed0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.254+0000 7f687cb48700 1 -- 172.21.15.22:0/1375618211 shutdown_connections 2023-07-06T01:20:20.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.254+0000 7f687cb48700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f68780adfd0 0x7f687815f220 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.254+0000 7f687cb48700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6860003a00 0x7f68780b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.254+0000 7f687cb48700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f685806ace0 0x7f685806d190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.254+0000 7f687cb48700 1 --2- 172.21.15.22:0/1375618211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68780adb60 0x7f687815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.254+0000 7f687cb48700 1 -- 172.21.15.22:0/1375618211 >> 172.21.15.22:0/1375618211 conn(0x7f68780b1890 msgr2=0x7f68780b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:20.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.254+0000 7f687cb48700 1 -- 172.21.15.22:0/1375618211 shutdown_connections 2023-07-06T01:20:20.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.254+0000 7f687cb48700 1 -- 172.21.15.22:0/1375618211 wait complete. 2023-07-06T01:20:20.268 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:20:20.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.391+0000 7f118c9cd700 1 Processor -- start 2023-07-06T01:20:20.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.391+0000 7f118c9cd700 1 -- start start 2023-07-06T01:20:20.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f118c9cd700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1188159360 0x7f118815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:20.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f118c9cd700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f11880cc660 0x7f11880ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:20.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f118c9cd700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11880cf230 con 0x7f1188070a20 2023-07-06T01:20:20.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f118c9cd700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118815df20 con 0x7f11880cc660 2023-07-06T01:20:20.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f118c9cd700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11880ccff0 con 0x7f1188159360 2023-07-06T01:20:20.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f1185d9b700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f11880cc660 0x7f11880ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:20.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f118659c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1188159360 0x7f118815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:20.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f1185d9b700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f11880cc660 0x7f11880ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:35936/0 (socket says 172.21.15.22:35936) 2023-07-06T01:20:20.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f1186d9d700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f1188070a20 0x7f1188070e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:40024/0 (socket says 172.21.15.22:40024) 2023-07-06T01:20:20.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f1185d9b700 1 -- 172.21.15.22:0/2696101886 learned_addr learned my addr 172.21.15.22:0/2696101886 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:20.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f118659c700 1 --2- 172.21.15.22:0/2696101886 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1188159360 0x7f118815d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:58350/0 (socket says 172.21.15.22:58350) 2023-07-06T01:20:20.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f118659c700 1 -- 172.21.15.22:0/2696101886 >> v2:172.21.15.103:3300/0 conn(0x7f11880cc660 msgr2=0x7f11880ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:20.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f118659c700 1 --2- 172.21.15.22:0/2696101886 >> v2:172.21.15.103:3300/0 conn(0x7f11880cc660 0x7f11880ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f118659c700 1 -- 172.21.15.22:0/2696101886 >> v1:172.21.15.103:6789/0 conn(0x7f1188070a20 legacy=0x7f1188070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:20.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f118659c700 1 -- 172.21.15.22:0/2696101886 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11880ce620 con 0x7f1188159360 2023-07-06T01:20:20.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.392+0000 7f1185d9b700 1 --2- 172.21.15.22:0/2696101886 >> v2:172.21.15.103:3300/0 conn(0x7f11880cc660 0x7f11880ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:20.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.393+0000 7f118659c700 1 --2- 172.21.15.22:0/2696101886 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1188159360 0x7f118815d9e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1178009840 tx=0x7f1178009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e41493b1f58a838b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:20.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.393+0000 7f118559a700 1 -- 172.21.15.22:0/2696101886 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178011830 con 0x7f1188159360 2023-07-06T01:20:20.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.393+0000 7f118559a700 1 -- 172.21.15.22:0/2696101886 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1178011990 con 0x7f1188159360 2023-07-06T01:20:20.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.393+0000 7f118559a700 1 -- 172.21.15.22:0/2696101886 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178011d20 con 0x7f1188159360 2023-07-06T01:20:20.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.393+0000 7f118c9cd700 1 -- 172.21.15.22:0/2696101886 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1188159360 msgr2=0x7f118815d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:20.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.393+0000 7f118c9cd700 1 --2- 172.21.15.22:0/2696101886 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1188159360 0x7f118815d9e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1178009840 tx=0x7f1178009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:20.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.393+0000 7f118c9cd700 1 -- 172.21.15.22:0/2696101886 shutdown_connections 2023-07-06T01:20:20.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.393+0000 7f118c9cd700 1 --2- 172.21.15.22:0/2696101886 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1188159360 0x7f118815d9e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.393+0000 7f118c9cd700 1 --2- 172.21.15.22:0/2696101886 >> v2:172.21.15.103:3300/0 conn(0x7f11880cc660 0x7f11880ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.393+0000 7f118c9cd700 1 -- 172.21.15.22:0/2696101886 >> 172.21.15.22:0/2696101886 conn(0x7f11880b1890 msgr2=0x7f11880b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:20.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.393+0000 7f118c9cd700 1 -- 172.21.15.22:0/2696101886 shutdown_connections 2023-07-06T01:20:20.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.393+0000 7f118c9cd700 1 -- 172.21.15.22:0/2696101886 wait complete. 2023-07-06T01:20:20.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.395+0000 7f118c9cd700 1 Processor -- start 2023-07-06T01:20:20.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.395+0000 7f118c9cd700 1 -- start start 2023-07-06T01:20:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.395+0000 7f118c9cd700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1188070a20 0x7f11880b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.395+0000 7f118c9cd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11880cc660 0x7f118814c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.395+0000 7f118c9cd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1188159360 0x7f118815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f118c9cd700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11880b5580 con 0x7f11880cc660 2023-07-06T01:20:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f118c9cd700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1188159fb0 con 0x7f1188159360 2023-07-06T01:20:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f118c9cd700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11880b45f0 con 0x7f1188070a20 2023-07-06T01:20:20.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f1185d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11880cc660 0x7f118814c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:20.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f1185d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11880cc660 0x7f118814c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47220/0 (socket says 172.21.15.22:47220) 2023-07-06T01:20:20.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f1185d9b700 1 -- 172.21.15.22:0/3716126182 learned_addr learned my addr 172.21.15.22:0/3716126182 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:20.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f118659c700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1188070a20 0x7f11880b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:20.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f1186d9d700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1188159360 0x7f118815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:20.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f1185d9b700 1 -- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1188070a20 msgr2=0x7f11880b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:20.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f1185d9b700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1188070a20 0x7f11880b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f1185d9b700 1 -- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1188159360 msgr2=0x7f118815bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:20.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f1185d9b700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1188159360 0x7f118815bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f1185d9b700 1 -- 172.21.15.22:0/3716126182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c009550 con 0x7f11880cc660 2023-07-06T01:20:20.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f1185d9b700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11880cc660 0x7f118814c320 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f117c00ebe0 tx=0x7f117c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:20.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f1176ffd700 1 -- 172.21.15.22:0/3716126182 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00c850 con 0x7f11880cc660 2023-07-06T01:20:20.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f1176ffd700 1 -- 172.21.15.22:0/3716126182 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c00c9b0 con 0x7f11880cc660 2023-07-06T01:20:20.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f1176ffd700 1 -- 172.21.15.22:0/3716126182 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00cd40 con 0x7f11880cc660 2023-07-06T01:20:20.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f118c9cd700 1 -- 172.21.15.22:0/3716126182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11780094a0 con 0x7f11880cc660 2023-07-06T01:20:20.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.396+0000 7f118c9cd700 1 -- 172.21.15.22:0/3716126182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11880b4bb0 con 0x7f11880cc660 2023-07-06T01:20:20.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.397+0000 7f118c9cd700 1 -- 172.21.15.22:0/3716126182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11880b4d40 con 0x7f11880cc660 2023-07-06T01:20:20.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.398+0000 7f1176ffd700 1 -- 172.21.15.22:0/3716126182 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f117c013470 con 0x7f11880cc660 2023-07-06T01:20:20.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.398+0000 7f1176ffd700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f116806ace0 0x7f116806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:20.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.398+0000 7f1176ffd700 1 -- 172.21.15.22:0/3716126182 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f117c0891c0 con 0x7f11880cc660 2023-07-06T01:20:20.405 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.399+0000 7f118659c700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f116806ace0 0x7f116806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:20.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.399+0000 7f118659c700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f116806ace0 0x7f116806d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1178000c00 tx=0x7f1178008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:20.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.403+0000 7f1176ffd700 1 -- 172.21.15.22:0/3716126182 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f117c054650 con 0x7f11880cc660 2023-07-06T01:20:20.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.600+0000 7f118c9cd700 1 -- 172.21.15.22:0/3716126182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11880744b0 con 0x7f11880cc660 2023-07-06T01:20:20.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.601+0000 7f1176ffd700 1 -- 172.21.15.22:0/3716126182 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+14569 (secure 0 0 0) 0x7f117c057c70 con 0x7f11880cc660 2023-07-06T01:20:20.603 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:20.603 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":30,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:20:17.173851+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:20:20.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.603+0000 7f118c9cd700 1 -- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f116806ace0 msgr2=0x7f116806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:20.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.603+0000 7f118c9cd700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f116806ace0 0x7f116806d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1178000c00 tx=0x7f1178008030 comp rx=0 tx=0).stop 2023-07-06T01:20:20.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.603+0000 7f118c9cd700 1 -- 172.21.15.22:0/3716126182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11880cc660 msgr2=0x7f118814c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:20.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.603+0000 7f118c9cd700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11880cc660 0x7f118814c320 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f117c00ebe0 tx=0x7f117c00eef0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.603+0000 7f118c9cd700 1 -- 172.21.15.22:0/3716126182 shutdown_connections 2023-07-06T01:20:20.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.603+0000 7f118c9cd700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f116806ace0 0x7f116806d190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.603+0000 7f118c9cd700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1188159360 0x7f118815bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.603+0000 7f118c9cd700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1188070a20 0x7f11880b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.603+0000 7f118c9cd700 1 --2- 172.21.15.22:0/3716126182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11880cc660 0x7f118814c320 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.603+0000 7f118c9cd700 1 -- 172.21.15.22:0/3716126182 >> 172.21.15.22:0/3716126182 conn(0x7f11880b1890 msgr2=0x7f118815cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:20.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.604+0000 7f118c9cd700 1 -- 172.21.15.22:0/3716126182 shutdown_connections 2023-07-06T01:20:20.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.604+0000 7f118c9cd700 1 -- 172.21.15.22:0/3716126182 wait complete. 2023-07-06T01:20:20.617 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:20:20.618 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:20:20.747 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.746+0000 7f928acf3700 1 Processor -- start 2023-07-06T01:20:20.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.746+0000 7f928acf3700 1 -- start start 2023-07-06T01:20:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f928acf3700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92840af2a0 0x7f928406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f928acf3700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f928406a000 0x7f928406a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f928acf3700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f92840cbc10 0x7f92840cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:20.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f928acf3700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928406a9e0 con 0x7f92840cbc10 2023-07-06T01:20:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f928acf3700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92840af670 con 0x7f928406a000 2023-07-06T01:20:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f928acf3700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928406d030 con 0x7f92840af2a0 2023-07-06T01:20:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f9289290700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f92840cbc10 0x7f92840cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f9289290700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f92840cbc10 0x7f92840cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47230/0 (socket says 172.21.15.22:47230) 2023-07-06T01:20:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f9289290700 1 -- 172.21.15.22:0/2734008758 learned_addr learned my addr 172.21.15.22:0/2734008758 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f9288a8f700 1 --2- 172.21.15.22:0/2734008758 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92840af2a0 0x7f928406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:20.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f9283fff700 1 --2- 172.21.15.22:0/2734008758 >> v2:172.21.15.103:3300/0 conn(0x7f928406a000 0x7f928406a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:20.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f9289290700 1 -- 172.21.15.22:0/2734008758 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92840af2a0 msgr2=0x7f928406caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:20.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f9289290700 1 --2- 172.21.15.22:0/2734008758 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92840af2a0 0x7f928406caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f9289290700 1 -- 172.21.15.22:0/2734008758 >> v2:172.21.15.103:3300/0 conn(0x7f928406a000 msgr2=0x7f928406a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:20.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f9289290700 1 --2- 172.21.15.22:0/2734008758 >> v2:172.21.15.103:3300/0 conn(0x7f928406a000 0x7f928406a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f9289290700 1 -- 172.21.15.22:0/2734008758 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92840cd560 con 0x7f92840cbc10 2023-07-06T01:20:20.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f9289290700 1 --2- 172.21.15.22:0/2734008758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92840cbc10 0x7f92840cc090 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f92840cd7f0 tx=0x7f927c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72cf8b05645b223 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:20.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.747+0000 7f92837fe700 1 -- 172.21.15.22:0/2734008758 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00e070 con 0x7f92840cbc10 2023-07-06T01:20:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.748+0000 7f92837fe700 1 -- 172.21.15.22:0/2734008758 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c00b8d0 con 0x7f92840cbc10 2023-07-06T01:20:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.748+0000 7f92837fe700 1 -- 172.21.15.22:0/2734008758 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00bc80 con 0x7f92840cbc10 2023-07-06T01:20:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.748+0000 7f928acf3700 1 -- 172.21.15.22:0/2734008758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92840cbc10 msgr2=0x7f92840cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.748+0000 7f928acf3700 1 --2- 172.21.15.22:0/2734008758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92840cbc10 0x7f92840cc090 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f92840cd7f0 tx=0x7f927c00b330 comp rx=0 tx=0).stop 2023-07-06T01:20:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.748+0000 7f928acf3700 1 -- 172.21.15.22:0/2734008758 shutdown_connections 2023-07-06T01:20:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.748+0000 7f928acf3700 1 --2- 172.21.15.22:0/2734008758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92840cbc10 0x7f92840cc090 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.748+0000 7f928acf3700 1 --2- 172.21.15.22:0/2734008758 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92840af2a0 0x7f928406caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.748+0000 7f928acf3700 1 --2- 172.21.15.22:0/2734008758 >> v2:172.21.15.103:3300/0 conn(0x7f928406a000 0x7f928406a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.748+0000 7f928acf3700 1 -- 172.21.15.22:0/2734008758 >> 172.21.15.22:0/2734008758 conn(0x7f92840b1890 msgr2=0x7f92840b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.748+0000 7f928acf3700 1 -- 172.21.15.22:0/2734008758 shutdown_connections 2023-07-06T01:20:20.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.748+0000 7f928acf3700 1 -- 172.21.15.22:0/2734008758 wait complete. 2023-07-06T01:20:20.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.749+0000 7f928acf3700 1 Processor -- start 2023-07-06T01:20:20.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.749+0000 7f928acf3700 1 -- start start 2023-07-06T01:20:20.756 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.750+0000 7f928acf3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f928406a000 0x7f92840b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:20.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.750+0000 7f928acf3700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92840af2a0 0x7f928415f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:20.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.750+0000 7f928acf3700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92840cbc10 0x7f92840cf310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:20.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.750+0000 7f928acf3700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928415fb90 con 0x7f928406a000 2023-07-06T01:20:20.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.750+0000 7f928acf3700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928415e5b0 con 0x7f92840af2a0 2023-07-06T01:20:20.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.750+0000 7f928acf3700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928415e890 con 0x7f92840cbc10 2023-07-06T01:20:20.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.750+0000 7f9288a8f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f928406a000 0x7f92840b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:20.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.750+0000 7f9288a8f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f928406a000 0x7f92840b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47232/0 (socket says 172.21.15.22:47232) 2023-07-06T01:20:20.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.750+0000 7f9288a8f700 1 -- 172.21.15.22:0/1968339287 learned_addr learned my addr 172.21.15.22:0/1968339287 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:20.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f9289290700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92840cbc10 0x7f92840cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:20.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f9283fff700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92840af2a0 0x7f928415f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:20.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f9288a8f700 1 -- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92840cbc10 msgr2=0x7f92840cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:20.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f9288a8f700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92840cbc10 0x7f92840cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f9288a8f700 1 -- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92840af2a0 msgr2=0x7f928415f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:20.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f9288a8f700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92840af2a0 0x7f928415f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f9288a8f700 1 -- 172.21.15.22:0/1968339287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c009b30 con 0x7f928406a000 2023-07-06T01:20:20.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f9283fff700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92840af2a0 0x7f928415f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:20.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f9289290700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92840cbc10 0x7f92840cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:20.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f9288a8f700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f928406a000 0x7f92840b40d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f927800e9b0 tx=0x7f927800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:20.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f92817fa700 1 -- 172.21.15.22:0/1968339287 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278009710 con 0x7f928406a000 2023-07-06T01:20:20.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f92817fa700 1 -- 172.21.15.22:0/1968339287 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927800c850 con 0x7f928406a000 2023-07-06T01:20:20.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f928acf3700 1 -- 172.21.15.22:0/1968339287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928415eb70 con 0x7f928406a000 2023-07-06T01:20:20.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f92817fa700 1 -- 172.21.15.22:0/1968339287 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927800cbc0 con 0x7f928406a000 2023-07-06T01:20:20.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.751+0000 7f928acf3700 1 -- 172.21.15.22:0/1968339287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928415f130 con 0x7f928406a000 2023-07-06T01:20:20.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.752+0000 7f928acf3700 1 -- 172.21.15.22:0/1968339287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928415ca90 con 0x7f928406a000 2023-07-06T01:20:20.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.752+0000 7f92817fa700 1 -- 172.21.15.22:0/1968339287 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f927800cd20 con 0x7f928406a000 2023-07-06T01:20:20.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.753+0000 7f92817fa700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f926806ace0 0x7f926806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:20.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.753+0000 7f92817fa700 1 -- 172.21.15.22:0/1968339287 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9278089140 con 0x7f928406a000 2023-07-06T01:20:20.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.753+0000 7f9283fff700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f926806ace0 0x7f926806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:20.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.753+0000 7f9283fff700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f926806ace0 0x7f926806d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f928415b140 tx=0x7f9270008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:20.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.757+0000 7f92817fa700 1 -- 172.21.15.22:0/1968339287 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f928415ca90 con 0x7f928406a000 2023-07-06T01:20:20.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.955+0000 7f928acf3700 1 -- 172.21.15.22:0/1968339287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f928415ca90 con 0x7f928406a000 2023-07-06T01:20:20.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.955+0000 7f92817fa700 1 -- 172.21.15.22:0/1968339287 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f928415ca90 con 0x7f928406a000 2023-07-06T01:20:20.958 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:20:20.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.957+0000 7f928acf3700 1 -- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f926806ace0 msgr2=0x7f926806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:20.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.958+0000 7f928acf3700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f926806ace0 0x7f926806d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f928415b140 tx=0x7f9270008040 comp rx=0 tx=0).stop 2023-07-06T01:20:20.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.958+0000 7f928acf3700 1 -- 172.21.15.22:0/1968339287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f928406a000 msgr2=0x7f92840b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:20.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.958+0000 7f928acf3700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f928406a000 0x7f92840b40d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f927800e9b0 tx=0x7f927800ed70 comp rx=0 tx=0).stop 2023-07-06T01:20:20.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.958+0000 7f928acf3700 1 -- 172.21.15.22:0/1968339287 shutdown_connections 2023-07-06T01:20:20.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.958+0000 7f928acf3700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92840cbc10 0x7f92840cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.958+0000 7f928acf3700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f928406a000 0x7f92840b40d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.958+0000 7f928acf3700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f926806ace0 0x7f926806d190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.958+0000 7f928acf3700 1 --2- 172.21.15.22:0/1968339287 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92840af2a0 0x7f928415f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:20.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.958+0000 7f928acf3700 1 -- 172.21.15.22:0/1968339287 >> 172.21.15.22:0/1968339287 conn(0x7f92840b1890 msgr2=0x7f92840ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:20.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.958+0000 7f928acf3700 1 -- 172.21.15.22:0/1968339287 shutdown_connections 2023-07-06T01:20:20.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:20.958+0000 7f928acf3700 1 -- 172.21.15.22:0/1968339287 wait complete. 2023-07-06T01:20:20.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:20:21.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.098+0000 7f1f8d954700 1 Processor -- start 2023-07-06T01:20:21.100 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.098+0000 7f1f8d954700 1 -- start start 2023-07-06T01:20:21.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f8d954700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1f880b3b70 0x7f1f880d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f8d954700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f880cebd0 con 0x7f1f880b36f0 2023-07-06T01:20:21.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f8d954700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f880cdfc0 con 0x7f1f880b40c0 2023-07-06T01:20:21.101 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f8d954700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8815cff0 con 0x7f1f880b3b70 2023-07-06T01:20:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f867fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1f880b3b70 0x7f1f880d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f86ffd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1f880b36f0 0x7f1f880cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:37984/0 (socket says 172.21.15.22:37984) 2023-07-06T01:20:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f867fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1f880b3b70 0x7f1f880d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47244/0 (socket says 172.21.15.22:47244) 2023-07-06T01:20:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f86ffd700 1 -- 172.21.15.22:0/1455045069 learned_addr learned my addr 172.21.15.22:0/1455045069 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f867fc700 1 -- 172.21.15.22:0/1455045069 >> v1:172.21.15.103:6789/0 conn(0x7f1f880b40c0 legacy=0x7f1f880b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f867fc700 1 -- 172.21.15.22:0/1455045069 >> v1:172.21.15.22:6789/0 conn(0x7f1f880b36f0 legacy=0x7f1f880cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.102 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f867fc700 1 -- 172.21.15.22:0/1455045069 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f880ce2a0 con 0x7f1f880b3b70 2023-07-06T01:20:21.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f85ffb700 1 -- 172.21.15.22:0/1455045069 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573659578 0 0) 0x7f1f880cebd0 con 0x7f1f880b36f0 2023-07-06T01:20:21.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f867fc700 1 --2- 172.21.15.22:0/1455045069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f880b3b70 0x7f1f880d0a90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009840 tx=0x7f1f78009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80810b70ae2a2f60 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:21.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.099+0000 7f1f85ffb700 1 -- 172.21.15.22:0/1455045069 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78004230 con 0x7f1f880b3b70 2023-07-06T01:20:21.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.100+0000 7f1f85ffb700 1 -- 172.21.15.22:0/1455045069 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f78004390 con 0x7f1f880b3b70 2023-07-06T01:20:21.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.100+0000 7f1f85ffb700 1 -- 172.21.15.22:0/1455045069 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f780036a0 con 0x7f1f880b3b70 2023-07-06T01:20:21.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.100+0000 7f1f8d954700 1 -- 172.21.15.22:0/1455045069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f880b3b70 msgr2=0x7f1f880d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.100+0000 7f1f8d954700 1 --2- 172.21.15.22:0/1455045069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f880b3b70 0x7f1f880d0a90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009840 tx=0x7f1f78009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:21.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.100+0000 7f1f8d954700 1 -- 172.21.15.22:0/1455045069 shutdown_connections 2023-07-06T01:20:21.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.100+0000 7f1f8d954700 1 --2- 172.21.15.22:0/1455045069 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f880b3b70 0x7f1f880d0a90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.100+0000 7f1f8d954700 1 -- 172.21.15.22:0/1455045069 >> 172.21.15.22:0/1455045069 conn(0x7f1f8809c2e0 msgr2=0x7f1f880d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:21.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.100+0000 7f1f8d954700 1 -- 172.21.15.22:0/1455045069 shutdown_connections 2023-07-06T01:20:21.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.100+0000 7f1f8d954700 1 -- 172.21.15.22:0/1455045069 wait complete. 2023-07-06T01:20:21.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.101+0000 7f1f8d954700 1 Processor -- start 2023-07-06T01:20:21.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.102+0000 7f1f8d954700 1 -- start start 2023-07-06T01:20:21.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.102+0000 7f1f8d954700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f880b36f0 0x7f1f8815fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.102+0000 7f1f8d954700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f880b3b70 0x7f1f88157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.102+0000 7f1f8d954700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f880b40c0 0x7f1f88158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.102+0000 7f1f8d954700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f880b8240 con 0x7f1f880b3b70 2023-07-06T01:20:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.102+0000 7f1f8d954700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8815f6c0 con 0x7f1f880b36f0 2023-07-06T01:20:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.102+0000 7f1f8d954700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8815eab0 con 0x7f1f880b40c0 2023-07-06T01:20:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.102+0000 7f1f867fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f880b3b70 0x7f1f88157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.102+0000 7f1f867fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f880b3b70 0x7f1f88157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47246/0 (socket says 172.21.15.22:47246) 2023-07-06T01:20:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.102+0000 7f1f867fc700 1 -- 172.21.15.22:0/1567277775 learned_addr learned my addr 172.21.15.22:0/1567277775 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:21.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.103+0000 7f1f86ffd700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f880b36f0 0x7f1f8815fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.103+0000 7f1f877fe700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f880b40c0 0x7f1f88158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.103+0000 7f1f867fc700 1 -- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f880b40c0 msgr2=0x7f1f88158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.103+0000 7f1f867fc700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f880b40c0 0x7f1f88158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.103+0000 7f1f867fc700 1 -- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f880b36f0 msgr2=0x7f1f8815fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.103+0000 7f1f867fc700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f880b36f0 0x7f1f8815fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.103+0000 7f1f867fc700 1 -- 172.21.15.22:0/1567277775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f780094a0 con 0x7f1f880b3b70 2023-07-06T01:20:21.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.103+0000 7f1f867fc700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f880b3b70 0x7f1f88157bc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1f88068130 tx=0x7f1f78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.103+0000 7f1f6ffff700 1 -- 172.21.15.22:0/1567277775 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78011630 con 0x7f1f880b3b70 2023-07-06T01:20:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.103+0000 7f1f6ffff700 1 -- 172.21.15.22:0/1567277775 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7801e070 con 0x7f1f880b3b70 2023-07-06T01:20:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.103+0000 7f1f8d954700 1 -- 172.21.15.22:0/1567277775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8815ed90 con 0x7f1f880b3b70 2023-07-06T01:20:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.103+0000 7f1f6ffff700 1 -- 172.21.15.22:0/1567277775 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f780119f0 con 0x7f1f880b3b70 2023-07-06T01:20:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.103+0000 7f1f8d954700 1 -- 172.21.15.22:0/1567277775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8815f350 con 0x7f1f880b3b70 2023-07-06T01:20:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.104+0000 7f1f8d954700 1 -- 172.21.15.22:0/1567277775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f880b4840 con 0x7f1f880b3b70 2023-07-06T01:20:21.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.104+0000 7f1f6ffff700 1 -- 172.21.15.22:0/1567277775 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f78011b50 con 0x7f1f880b3b70 2023-07-06T01:20:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.105+0000 7f1f6ffff700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1f7406ace0 0x7f1f7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.105+0000 7f1f6ffff700 1 -- 172.21.15.22:0/1567277775 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1f7808a490 con 0x7f1f880b3b70 2023-07-06T01:20:21.111 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.105+0000 7f1f86ffd700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1f7406ace0 0x7f1f7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.105+0000 7f1f86ffd700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1f7406ace0 0x7f1f7406d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c000f60 tx=0x7f1f7c007d90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:21.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.109+0000 7f1f6ffff700 1 -- 172.21.15.22:0/1567277775 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1f78055920 con 0x7f1f880b3b70 2023-07-06T01:20:21.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.305+0000 7f1f8d954700 1 -- 172.21.15.22:0/1567277775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f880744b0 con 0x7f1f880b3b70 2023-07-06T01:20:21.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.306+0000 7f1f6ffff700 1 -- 172.21.15.22:0/1567277775 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f78058f40 con 0x7f1f880b3b70 2023-07-06T01:20:21.308 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:20:21.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.308+0000 7f1f8d954700 1 -- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1f7406ace0 msgr2=0x7f1f7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.308+0000 7f1f8d954700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1f7406ace0 0x7f1f7406d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c000f60 tx=0x7f1f7c007d90 comp rx=0 tx=0).stop 2023-07-06T01:20:21.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.308+0000 7f1f8d954700 1 -- 172.21.15.22:0/1567277775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f880b3b70 msgr2=0x7f1f88157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.308+0000 7f1f8d954700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f880b3b70 0x7f1f88157bc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1f88068130 tx=0x7f1f78004750 comp rx=0 tx=0).stop 2023-07-06T01:20:21.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.308+0000 7f1f8d954700 1 -- 172.21.15.22:0/1567277775 shutdown_connections 2023-07-06T01:20:21.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.308+0000 7f1f8d954700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f880b40c0 0x7f1f88158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.308+0000 7f1f8d954700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1f7406ace0 0x7f1f7406d190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.308+0000 7f1f8d954700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f880b36f0 0x7f1f8815fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.308+0000 7f1f8d954700 1 --2- 172.21.15.22:0/1567277775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f880b3b70 0x7f1f88157bc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.308+0000 7f1f8d954700 1 -- 172.21.15.22:0/1567277775 >> 172.21.15.22:0/1567277775 conn(0x7f1f8809c2e0 msgr2=0x7f1f881e52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.308+0000 7f1f8d954700 1 -- 172.21.15.22:0/1567277775 shutdown_connections 2023-07-06T01:20:21.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.308+0000 7f1f8d954700 1 -- 172.21.15.22:0/1567277775 wait complete. 2023-07-06T01:20:21.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T01:20:21.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.445+0000 7f98cd796700 1 Processor -- start 2023-07-06T01:20:21.447 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.445+0000 7f98cd796700 1 -- start start 2023-07-06T01:20:21.448 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.446+0000 7f98cd796700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98c80b3b70 0x7f98c80d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.446+0000 7f98cd796700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f98c80b40c0 0x7f98c80d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.446+0000 7f98cd796700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c80cebd0 con 0x7f98c80b36f0 2023-07-06T01:20:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.446+0000 7f98cd796700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c815cff0 con 0x7f98c80b40c0 2023-07-06T01:20:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.446+0000 7f98cd796700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c80cdfc0 con 0x7f98c80b3b70 2023-07-06T01:20:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.446+0000 7f98c67fc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98c80b3b70 0x7f98c80d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.446+0000 7f98c67fc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98c80b3b70 0x7f98c80d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:58392/0 (socket says 172.21.15.22:58392) 2023-07-06T01:20:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.446+0000 7f98c67fc700 1 -- 172.21.15.22:0/3859665828 learned_addr learned my addr 172.21.15.22:0/3859665828 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:21.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.446+0000 7f98c77fe700 1 --2- 172.21.15.22:0/3859665828 >> v2:172.21.15.103:3300/0 conn(0x7f98c80b40c0 0x7f98c80d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.447+0000 7f98c67fc700 1 -- 172.21.15.22:0/3859665828 >> v2:172.21.15.103:3300/0 conn(0x7f98c80b40c0 msgr2=0x7f98c80d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:21.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.447+0000 7f98c67fc700 1 --2- 172.21.15.22:0/3859665828 >> v2:172.21.15.103:3300/0 conn(0x7f98c80b40c0 0x7f98c80d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.447+0000 7f98c67fc700 1 -- 172.21.15.22:0/3859665828 >> v1:172.21.15.103:6789/0 conn(0x7f98c80b36f0 legacy=0x7f98c80cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.447+0000 7f98c67fc700 1 -- 172.21.15.22:0/3859665828 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c80ce2a0 con 0x7f98c80b3b70 2023-07-06T01:20:21.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.447+0000 7f98c77fe700 1 --2- 172.21.15.22:0/3859665828 >> v2:172.21.15.103:3300/0 conn(0x7f98c80b40c0 0x7f98c80d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:21.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.447+0000 7f98c67fc700 1 --2- 172.21.15.22:0/3859665828 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98c80b3b70 0x7f98c80d0a90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f98b80098f0 tx=0x7f98b8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ece2e9dc1746f2b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:21.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.447+0000 7f98c5ffb700 1 -- 172.21.15.22:0/3859665828 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b8011830 con 0x7f98c80b3b70 2023-07-06T01:20:21.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.447+0000 7f98c5ffb700 1 -- 172.21.15.22:0/3859665828 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b8011990 con 0x7f98c80b3b70 2023-07-06T01:20:21.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.447+0000 7f98c5ffb700 1 -- 172.21.15.22:0/3859665828 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b8011d20 con 0x7f98c80b3b70 2023-07-06T01:20:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.447+0000 7f98cd796700 1 -- 172.21.15.22:0/3859665828 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98c80b3b70 msgr2=0x7f98c80d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.447+0000 7f98cd796700 1 --2- 172.21.15.22:0/3859665828 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98c80b3b70 0x7f98c80d0a90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f98b80098f0 tx=0x7f98b8009c00 comp rx=0 tx=0).stop 2023-07-06T01:20:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.448+0000 7f98cd796700 1 -- 172.21.15.22:0/3859665828 shutdown_connections 2023-07-06T01:20:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.448+0000 7f98cd796700 1 --2- 172.21.15.22:0/3859665828 >> v2:172.21.15.103:3300/0 conn(0x7f98c80b40c0 0x7f98c80d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.448+0000 7f98cd796700 1 --2- 172.21.15.22:0/3859665828 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98c80b3b70 0x7f98c80d0a90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.448+0000 7f98cd796700 1 -- 172.21.15.22:0/3859665828 >> 172.21.15.22:0/3859665828 conn(0x7f98c809c2e0 msgr2=0x7f98c80d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.448+0000 7f98cd796700 1 -- 172.21.15.22:0/3859665828 shutdown_connections 2023-07-06T01:20:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.448+0000 7f98cd796700 1 -- 172.21.15.22:0/3859665828 wait complete. 2023-07-06T01:20:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.449+0000 7f98cd796700 1 Processor -- start 2023-07-06T01:20:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.449+0000 7f98cd796700 1 -- start start 2023-07-06T01:20:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98cd796700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98c80b36f0 0x7f98c80b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98cd796700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98c80b3b70 0x7f98c80c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98cd796700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98c80b40c0 0x7f98c8158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98cd796700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c80c8020 con 0x7f98c80b36f0 2023-07-06T01:20:21.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98cd796700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c815d130 con 0x7f98c80b40c0 2023-07-06T01:20:21.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98cd796700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c8158980 con 0x7f98c80b3b70 2023-07-06T01:20:21.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98c6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98c80b36f0 0x7f98c80b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98c6ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98c80b36f0 0x7f98c80b6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47260/0 (socket says 172.21.15.22:47260) 2023-07-06T01:20:21.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98c6ffd700 1 -- 172.21.15.22:0/1092378576 learned_addr learned my addr 172.21.15.22:0/1092378576 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:21.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98c67fc700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98c80b3b70 0x7f98c80c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98c77fe700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98c80b40c0 0x7f98c8158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98c6ffd700 1 -- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98c80b3b70 msgr2=0x7f98c80c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98c6ffd700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98c80b3b70 0x7f98c80c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98c6ffd700 1 -- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98c80b40c0 msgr2=0x7f98c8158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98c6ffd700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98c80b40c0 0x7f98c8158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98c6ffd700 1 -- 172.21.15.22:0/1092378576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0009a70 con 0x7f98c80b36f0 2023-07-06T01:20:21.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98c67fc700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98c80b3b70 0x7f98c80c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:21.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98c6ffd700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98c80b36f0 0x7f98c80b6370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00c4e0 tx=0x7f98bc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:21.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.450+0000 7f98af7fe700 1 -- 172.21.15.22:0/1092378576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0052d0 con 0x7f98c80b36f0 2023-07-06T01:20:21.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.451+0000 7f98af7fe700 1 -- 172.21.15.22:0/1092378576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc005430 con 0x7f98c80b36f0 2023-07-06T01:20:21.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.451+0000 7f98cd796700 1 -- 172.21.15.22:0/1092378576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b8009550 con 0x7f98c80b36f0 2023-07-06T01:20:21.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.451+0000 7f98af7fe700 1 -- 172.21.15.22:0/1092378576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0038c0 con 0x7f98c80b36f0 2023-07-06T01:20:21.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.451+0000 7f98cd796700 1 -- 172.21.15.22:0/1092378576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c80c85a0 con 0x7f98c80b36f0 2023-07-06T01:20:21.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.452+0000 7f98cd796700 1 -- 172.21.15.22:0/1092378576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c80b4840 con 0x7f98c80b36f0 2023-07-06T01:20:21.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.452+0000 7f98af7fe700 1 -- 172.21.15.22:0/1092378576 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f98bc003a20 con 0x7f98c80b36f0 2023-07-06T01:20:21.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.452+0000 7f98af7fe700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f98b406ace0 0x7f98b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.453+0000 7f98af7fe700 1 -- 172.21.15.22:0/1092378576 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f98bc0884b0 con 0x7f98c80b36f0 2023-07-06T01:20:21.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.453+0000 7f98c67fc700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f98b406ace0 0x7f98b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.453+0000 7f98c67fc700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f98b406ace0 0x7f98b406d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009c40 tx=0x7f98b8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:21.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.457+0000 7f98af7fe700 1 -- 172.21.15.22:0/1092378576 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f98bc0538b0 con 0x7f98c80b36f0 2023-07-06T01:20:21.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.658+0000 7f98cd796700 1 -- 172.21.15.22:0/1092378576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f98c80744b0 con 0x7f98c80b36f0 2023-07-06T01:20:21.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.658+0000 7f98af7fe700 1 -- 172.21.15.22:0/1092378576 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+126 (secure 0 0 0) 0x7f98bc056ed0 con 0x7f98c80b36f0 2023-07-06T01:20:21.660 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:21.661 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2023-07-06T01:20:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2023-07-06T01:20:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": "rbd" 2023-07-06T01:20:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:20:21.661 INFO:teuthology.orchestra.run.smithi022.stdout: { 2023-07-06T01:20:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 2, 2023-07-06T01:20:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2023-07-06T01:20:21.662 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:20:21.662 INFO:teuthology.orchestra.run.smithi022.stdout:] 2023-07-06T01:20:21.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.660+0000 7f98cd796700 1 -- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f98b406ace0 msgr2=0x7f98b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.660+0000 7f98cd796700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f98b406ace0 0x7f98b406d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009c40 tx=0x7f98b8008030 comp rx=0 tx=0).stop 2023-07-06T01:20:21.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.660+0000 7f98cd796700 1 -- 172.21.15.22:0/1092378576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98c80b36f0 msgr2=0x7f98c80b6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.660+0000 7f98cd796700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98c80b36f0 0x7f98c80b6370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00c4e0 tx=0x7f98bc00c7f0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.663 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.660+0000 7f98cd796700 1 -- 172.21.15.22:0/1092378576 shutdown_connections 2023-07-06T01:20:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.660+0000 7f98cd796700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f98b406ace0 0x7f98b406d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.660+0000 7f98cd796700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98c80b40c0 0x7f98c8158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.660+0000 7f98cd796700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98c80b36f0 0x7f98c80b6370 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.664 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.660+0000 7f98cd796700 1 --2- 172.21.15.22:0/1092378576 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98c80b3b70 0x7f98c80c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.660+0000 7f98cd796700 1 -- 172.21.15.22:0/1092378576 >> 172.21.15.22:0/1092378576 conn(0x7f98c809c2e0 msgr2=0x7f98c806a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.661+0000 7f98cd796700 1 -- 172.21.15.22:0/1092378576 shutdown_connections 2023-07-06T01:20:21.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.661+0000 7f98cd796700 1 -- 172.21.15.22:0/1092378576 wait complete. 2023-07-06T01:20:21.674 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T01:20:21.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T01:20:21.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.803+0000 7f5404041700 1 Processor -- start 2023-07-06T01:20:21.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.803+0000 7f5404041700 1 -- start start 2023-07-06T01:20:21.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f5404041700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f53fc15a690 0x7f53fc0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f5404041700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f53fc0cee60 0x7f53fc06a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f5404041700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53fc0cd640 con 0x7f53fc0c9a40 2023-07-06T01:20:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f5404041700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53fc0cd920 con 0x7f53fc15a690 2023-07-06T01:20:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f5404041700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53fc0cdc00 con 0x7f53fc0cee60 2023-07-06T01:20:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f54015dc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f53fc15a690 0x7f53fc0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f5401ddd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f53fc0c9a40 0x7f53fc0c9e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38012/0 (socket says 172.21.15.22:38012) 2023-07-06T01:20:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f5401ddd700 1 -- 172.21.15.22:0/4168397067 learned_addr learned my addr 172.21.15.22:0/4168397067 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f54015dc700 1 -- 172.21.15.22:0/4168397067 >> v2:172.21.15.103:3300/0 conn(0x7f53fc0cee60 msgr2=0x7f53fc06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f54015dc700 1 --2- 172.21.15.22:0/4168397067 >> v2:172.21.15.103:3300/0 conn(0x7f53fc0cee60 0x7f53fc06a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f54015dc700 1 -- 172.21.15.22:0/4168397067 >> v1:172.21.15.22:6789/0 conn(0x7f53fc0c9a40 legacy=0x7f53fc0c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f5400ddb700 1 -- 172.21.15.22:0/4168397067 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716433893 0 0) 0x7f53fc0cd640 con 0x7f53fc0c9a40 2023-07-06T01:20:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f54015dc700 1 -- 172.21.15.22:0/4168397067 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53fc0cdee0 con 0x7f53fc15a690 2023-07-06T01:20:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f54015dc700 1 --2- 172.21.15.22:0/4168397067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53fc15a690 0x7f53fc0ce920 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0cff50 tx=0x7f53ec009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13dbde22c03a7524 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.804+0000 7f5400ddb700 1 -- 172.21.15.22:0/4168397067 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec011830 con 0x7f53fc15a690 2023-07-06T01:20:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.805+0000 7f5400ddb700 1 -- 172.21.15.22:0/4168397067 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ec011990 con 0x7f53fc15a690 2023-07-06T01:20:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.805+0000 7f5400ddb700 1 -- 172.21.15.22:0/4168397067 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec011d20 con 0x7f53fc15a690 2023-07-06T01:20:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.805+0000 7f5404041700 1 -- 172.21.15.22:0/4168397067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53fc15a690 msgr2=0x7f53fc0ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.805+0000 7f5404041700 1 --2- 172.21.15.22:0/4168397067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53fc15a690 0x7f53fc0ce920 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0cff50 tx=0x7f53ec009b20 comp rx=0 tx=0).stop 2023-07-06T01:20:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.805+0000 7f5404041700 1 -- 172.21.15.22:0/4168397067 shutdown_connections 2023-07-06T01:20:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.805+0000 7f5404041700 1 --2- 172.21.15.22:0/4168397067 >> v2:172.21.15.103:3300/0 conn(0x7f53fc0cee60 0x7f53fc06a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.805+0000 7f5404041700 1 --2- 172.21.15.22:0/4168397067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53fc15a690 0x7f53fc0ce920 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.805+0000 7f5404041700 1 -- 172.21.15.22:0/4168397067 >> 172.21.15.22:0/4168397067 conn(0x7f53fc0ccb00 msgr2=0x7f53fc0cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.805+0000 7f5404041700 1 -- 172.21.15.22:0/4168397067 shutdown_connections 2023-07-06T01:20:21.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.805+0000 7f5404041700 1 -- 172.21.15.22:0/4168397067 wait complete. 2023-07-06T01:20:21.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.807+0000 7f5404041700 1 Processor -- start 2023-07-06T01:20:21.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.807+0000 7f5404041700 1 -- start start 2023-07-06T01:20:21.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f5404041700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53fc0c9a40 0x7f53fc0c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f5404041700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53fc0cee60 0x7f53fc0aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f5404041700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53fc15a690 0x7f53fc14cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f5404041700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc15c6f0 con 0x7f53fc0cee60 2023-07-06T01:20:21.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f5404041700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc15bdc0 con 0x7f53fc15a690 2023-07-06T01:20:21.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f5404041700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc15c0a0 con 0x7f53fc0c9a40 2023-07-06T01:20:21.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f54015dc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53fc0cee60 0x7f53fc0aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f54015dc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53fc0cee60 0x7f53fc0aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47276/0 (socket says 172.21.15.22:47276) 2023-07-06T01:20:21.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f54015dc700 1 -- 172.21.15.22:0/3822741324 learned_addr learned my addr 172.21.15.22:0/3822741324 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:21.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f5401ddd700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53fc0c9a40 0x7f53fc0c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f54025de700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53fc15a690 0x7f53fc14cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f54015dc700 1 -- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53fc0c9a40 msgr2=0x7f53fc0c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f54015dc700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53fc0c9a40 0x7f53fc0c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f54015dc700 1 -- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53fc15a690 msgr2=0x7f53fc14cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:21.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f54015dc700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53fc15a690 0x7f53fc14cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f54015dc700 1 -- 172.21.15.22:0/3822741324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec0094a0 con 0x7f53fc0cee60 2023-07-06T01:20:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f5401ddd700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53fc0c9a40 0x7f53fc0c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.808+0000 7f54015dc700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53fc0cee60 0x7f53fc0aee50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f53ec000c00 tx=0x7f53ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:21.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.809+0000 7f53f27fc700 1 -- 172.21.15.22:0/3822741324 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec0038a0 con 0x7f53fc0cee60 2023-07-06T01:20:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.809+0000 7f53f27fc700 1 -- 172.21.15.22:0/3822741324 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ec01f070 con 0x7f53fc0cee60 2023-07-06T01:20:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.809+0000 7f5404041700 1 -- 172.21.15.22:0/3822741324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53fc15c380 con 0x7f53fc0cee60 2023-07-06T01:20:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.809+0000 7f53f27fc700 1 -- 172.21.15.22:0/3822741324 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec003c20 con 0x7f53fc0cee60 2023-07-06T01:20:21.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.809+0000 7f5404041700 1 -- 172.21.15.22:0/3822741324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53fc14d0a0 con 0x7f53fc0cee60 2023-07-06T01:20:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.810+0000 7f5404041700 1 -- 172.21.15.22:0/3822741324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53fc14d260 con 0x7f53fc0cee60 2023-07-06T01:20:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.810+0000 7f53f27fc700 1 -- 172.21.15.22:0/3822741324 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f53ec01a430 con 0x7f53fc0cee60 2023-07-06T01:20:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.811+0000 7f53f27fc700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f53e006ace0 0x7f53e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:21.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.811+0000 7f53f27fc700 1 -- 172.21.15.22:0/3822741324 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f53ec021070 con 0x7f53fc0cee60 2023-07-06T01:20:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.811+0000 7f5401ddd700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f53e006ace0 0x7f53e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.811+0000 7f5401ddd700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f53e006ace0 0x7f53e006d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f53f8000f60 tx=0x7f53f8007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:21.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:21.815+0000 7f53f27fc700 1 -- 172.21.15.22:0/3822741324 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f53fc14d260 con 0x7f53fc0cee60 2023-07-06T01:20:22.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.012+0000 7f5404041700 1 -- 172.21.15.22:0/3822741324 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f53fc14d260 con 0x7f53fc0cee60 2023-07-06T01:20:22.827 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.826+0000 7f53f27fc700 1 -- 172.21.15.22:0/3822741324 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f53fc14d260 con 0x7f53fc0cee60 2023-07-06T01:20:22.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.831+0000 7f5404041700 1 -- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f53e006ace0 msgr2=0x7f53e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:22.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.831+0000 7f5404041700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f53e006ace0 0x7f53e006d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f53f8000f60 tx=0x7f53f8007ca0 comp rx=0 tx=0).stop 2023-07-06T01:20:22.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.831+0000 7f5404041700 1 -- 172.21.15.22:0/3822741324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53fc0cee60 msgr2=0x7f53fc0aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:22.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.831+0000 7f5404041700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53fc0cee60 0x7f53fc0aee50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f53ec000c00 tx=0x7f53ec004750 comp rx=0 tx=0).stop 2023-07-06T01:20:22.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.831+0000 7f5404041700 1 -- 172.21.15.22:0/3822741324 shutdown_connections 2023-07-06T01:20:22.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.831+0000 7f5404041700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f53e006ace0 0x7f53e006d190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:22.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.831+0000 7f5404041700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53fc15a690 0x7f53fc14cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:22.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.831+0000 7f5404041700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53fc0c9a40 0x7f53fc0c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:22.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.831+0000 7f5404041700 1 --2- 172.21.15.22:0/3822741324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53fc0cee60 0x7f53fc0aee50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:22.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.831+0000 7f5404041700 1 -- 172.21.15.22:0/3822741324 >> 172.21.15.22:0/3822741324 conn(0x7f53fc0ccb00 msgr2=0x7f53fc156830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:22.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.831+0000 7f5404041700 1 -- 172.21.15.22:0/3822741324 shutdown_connections 2023-07-06T01:20:22.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.831+0000 7f5404041700 1 -- 172.21.15.22:0/3822741324 wait complete. 2023-07-06T01:20:22.835 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2023-07-06T01:20:22.848 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T01:20:22.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.985+0000 7f8f99035700 1 Processor -- start 2023-07-06T01:20:22.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.985+0000 7f8f99035700 1 -- start start 2023-07-06T01:20:22.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f99035700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8f9406a000 0x7f8f94160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:22.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f99035700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f940cf0c0 con 0x7f8f9406c1f0 2023-07-06T01:20:22.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f99035700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9406a5a0 con 0x7f8f940cecf0 2023-07-06T01:20:22.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f99035700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f9415ce00 con 0x7f8f9406a000 2023-07-06T01:20:22.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f9259c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8f9406a000 0x7f8f94160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:22.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f9359e700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f8f9406c1f0 0x7f8f9406c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38028/0 (socket says 172.21.15.22:38028) 2023-07-06T01:20:22.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f9359e700 1 -- 172.21.15.22:0/2048223177 learned_addr learned my addr 172.21.15.22:0/2048223177 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:22.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f9259c700 1 -- 172.21.15.22:0/2048223177 >> v1:172.21.15.103:6789/0 conn(0x7f8f940cecf0 legacy=0x7f8f9406b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:22.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f9259c700 1 -- 172.21.15.22:0/2048223177 >> v1:172.21.15.22:6789/0 conn(0x7f8f9406c1f0 legacy=0x7f8f9406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:22.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f9259c700 1 -- 172.21.15.22:0/2048223177 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9406a880 con 0x7f8f9406a000 2023-07-06T01:20:22.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f9259c700 1 --2- 172.21.15.22:0/2048223177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f9406a000 0x7f8f94160280 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009840 tx=0x7f8f88009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e223adf7a062623 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:22.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f91d9b700 1 -- 172.21.15.22:0/2048223177 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88004230 con 0x7f8f9406a000 2023-07-06T01:20:22.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f91d9b700 1 -- 172.21.15.22:0/2048223177 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f88004390 con 0x7f8f9406a000 2023-07-06T01:20:22.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.986+0000 7f8f91d9b700 1 -- 172.21.15.22:0/2048223177 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f880036a0 con 0x7f8f9406a000 2023-07-06T01:20:22.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.987+0000 7f8f99035700 1 -- 172.21.15.22:0/2048223177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f9406a000 msgr2=0x7f8f94160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:22.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.987+0000 7f8f99035700 1 --2- 172.21.15.22:0/2048223177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f9406a000 0x7f8f94160280 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009840 tx=0x7f8f88009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:22.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.987+0000 7f8f99035700 1 -- 172.21.15.22:0/2048223177 shutdown_connections 2023-07-06T01:20:22.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.987+0000 7f8f99035700 1 --2- 172.21.15.22:0/2048223177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f9406a000 0x7f8f94160280 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:22.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.987+0000 7f8f99035700 1 -- 172.21.15.22:0/2048223177 >> 172.21.15.22:0/2048223177 conn(0x7f8f940ccb80 msgr2=0x7f8f940cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:22.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.987+0000 7f8f99035700 1 -- 172.21.15.22:0/2048223177 shutdown_connections 2023-07-06T01:20:22.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.987+0000 7f8f99035700 1 -- 172.21.15.22:0/2048223177 wait complete. 2023-07-06T01:20:22.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.988+0000 7f8f99035700 1 Processor -- start 2023-07-06T01:20:22.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.988+0000 7f8f99035700 1 -- start start 2023-07-06T01:20:22.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.989+0000 7f8f99035700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f9406a000 0x7f8f94156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:22.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.989+0000 7f8f99035700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f9406c1f0 0x7f8f9415e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:22.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.989+0000 7f8f99035700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f940cecf0 0x7f8f9415e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:22.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.989+0000 7f8f99035700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f940b4dd0 con 0x7f8f9406a000 2023-07-06T01:20:22.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.989+0000 7f8f99035700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f940c5c30 con 0x7f8f940cecf0 2023-07-06T01:20:22.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.989+0000 7f8f99035700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f940b5290 con 0x7f8f9406c1f0 2023-07-06T01:20:22.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.989+0000 7f8f92d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f9406a000 0x7f8f94156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:22.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.989+0000 7f8f92d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f9406a000 0x7f8f94156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47288/0 (socket says 172.21.15.22:47288) 2023-07-06T01:20:22.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.989+0000 7f8f92d9d700 1 -- 172.21.15.22:0/633899208 learned_addr learned my addr 172.21.15.22:0/633899208 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:22.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.989+0000 7f8f9359e700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f940cecf0 0x7f8f9415e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:22.994 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.989+0000 7f8f9259c700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f9406c1f0 0x7f8f9415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:22.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.990+0000 7f8f92d9d700 1 -- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f9406c1f0 msgr2=0x7f8f9415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:22.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.990+0000 7f8f92d9d700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f9406c1f0 0x7f8f9415e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:22.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.990+0000 7f8f92d9d700 1 -- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f940cecf0 msgr2=0x7f8f9415e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:22.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.990+0000 7f8f92d9d700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f940cecf0 0x7f8f9415e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:22.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.990+0000 7f8f92d9d700 1 -- 172.21.15.22:0/633899208 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f880094a0 con 0x7f8f9406a000 2023-07-06T01:20:22.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.990+0000 7f8f92d9d700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f9406a000 0x7f8f94156530 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400c580 tx=0x7f8f8400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:22.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.990+0000 7f8f7b7fe700 1 -- 172.21.15.22:0/633899208 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f84011790 con 0x7f8f9406a000 2023-07-06T01:20:22.995 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.990+0000 7f8f7b7fe700 1 -- 172.21.15.22:0/633899208 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f840118f0 con 0x7f8f9406a000 2023-07-06T01:20:22.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.990+0000 7f8f99035700 1 -- 172.21.15.22:0/633899208 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f940c55b0 con 0x7f8f9406a000 2023-07-06T01:20:22.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.990+0000 7f8f7b7fe700 1 -- 172.21.15.22:0/633899208 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f84011ca0 con 0x7f8f9406a000 2023-07-06T01:20:22.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.990+0000 7f8f99035700 1 -- 172.21.15.22:0/633899208 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f940c5850 con 0x7f8f9406a000 2023-07-06T01:20:22.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.991+0000 7f8f99035700 1 -- 172.21.15.22:0/633899208 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f940c5d70 con 0x7f8f9406a000 2023-07-06T01:20:22.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.991+0000 7f8f7b7fe700 1 -- 172.21.15.22:0/633899208 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8f8401c460 con 0x7f8f9406a000 2023-07-06T01:20:22.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.992+0000 7f8f7b7fe700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8f8006ace0 0x7f8f8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:22.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.992+0000 7f8f7b7fe700 1 -- 172.21.15.22:0/633899208 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f8f84089060 con 0x7f8f9406a000 2023-07-06T01:20:22.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.992+0000 7f8f9259c700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8f8006ace0 0x7f8f8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:22.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.993+0000 7f8f9259c700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8f8006ace0 0x7f8f8006d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f880090b0 tx=0x7f8f88008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:22.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:22.996+0000 7f8f7b7fe700 1 -- 172.21.15.22:0/633899208 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8f84054300 con 0x7f8f9406a000 2023-07-06T01:20:23.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.194+0000 7f8f99035700 1 -- 172.21.15.22:0/633899208 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8f940744b0 con 0x7f8f9406a000 2023-07-06T01:20:23.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.831+0000 7f8f7b7fe700 1 -- 172.21.15.22:0/633899208 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8f84057920 con 0x7f8f9406a000 2023-07-06T01:20:23.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.836+0000 7f8f99035700 1 -- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8f8006ace0 msgr2=0x7f8f8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:23.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.836+0000 7f8f99035700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8f8006ace0 0x7f8f8006d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f880090b0 tx=0x7f8f88008030 comp rx=0 tx=0).stop 2023-07-06T01:20:23.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.836+0000 7f8f99035700 1 -- 172.21.15.22:0/633899208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f9406a000 msgr2=0x7f8f94156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:23.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.836+0000 7f8f99035700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f9406a000 0x7f8f94156530 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400c580 tx=0x7f8f8400c940 comp rx=0 tx=0).stop 2023-07-06T01:20:23.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.836+0000 7f8f99035700 1 -- 172.21.15.22:0/633899208 shutdown_connections 2023-07-06T01:20:23.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.836+0000 7f8f99035700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8f8006ace0 0x7f8f8006d190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:23.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.836+0000 7f8f99035700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f940cecf0 0x7f8f9415e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:23.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.836+0000 7f8f99035700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f9406a000 0x7f8f94156530 secure :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400c580 tx=0x7f8f8400c940 comp rx=0 tx=0).stop 2023-07-06T01:20:23.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.836+0000 7f8f99035700 1 --2- 172.21.15.22:0/633899208 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8f9406c1f0 0x7f8f9415e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:23.839 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.836+0000 7f8f99035700 1 -- 172.21.15.22:0/633899208 >> 172.21.15.22:0/633899208 conn(0x7f8f940ccb80 msgr2=0x7f8f9406b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:23.840 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.838+0000 7f8f99035700 1 -- 172.21.15.22:0/633899208 shutdown_connections 2023-07-06T01:20:23.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.840+0000 7f8f99035700 1 -- 172.21.15.22:0/633899208 wait complete. 2023-07-06T01:20:23.851 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2023-07-06T01:20:23.866 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T01:20:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.998+0000 7fda59820700 1 Processor -- start 2023-07-06T01:20:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.998+0000 7fda59820700 1 -- start start 2023-07-06T01:20:24.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda59820700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fda540b36f0 0x7fda540cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:24.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda59820700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda540cf5a0 0x7fda540d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:24.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda59820700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda540cebd0 con 0x7fda540b3b70 2023-07-06T01:20:24.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda59820700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5415cff0 con 0x7fda540b36f0 2023-07-06T01:20:24.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda59820700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda540cdfc0 con 0x7fda540cf5a0 2023-07-06T01:20:24.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda52ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fda540b36f0 0x7fda540cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:24.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda52ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fda540b36f0 0x7fda540cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47296/0 (socket says 172.21.15.22:47296) 2023-07-06T01:20:24.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda52ffd700 1 -- 172.21.15.22:0/399198787 learned_addr learned my addr 172.21.15.22:0/399198787 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:24.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda527fc700 1 --2- 172.21.15.22:0/399198787 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda540cf5a0 0x7fda540d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:24.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda52ffd700 1 -- 172.21.15.22:0/399198787 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda540cf5a0 msgr2=0x7fda540d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:24.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda52ffd700 1 --2- 172.21.15.22:0/399198787 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda540cf5a0 0x7fda540d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:24.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda52ffd700 1 -- 172.21.15.22:0/399198787 >> v1:172.21.15.103:6789/0 conn(0x7fda540b3b70 legacy=0x7fda540b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:24.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda52ffd700 1 -- 172.21.15.22:0/399198787 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda540ce2a0 con 0x7fda540b36f0 2023-07-06T01:20:24.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda527fc700 1 --2- 172.21.15.22:0/399198787 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda540cf5a0 0x7fda540d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:24.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:23.999+0000 7fda52ffd700 1 --2- 172.21.15.22:0/399198787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda540b36f0 0x7fda540cf060 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fda480098f0 tx=0x7fda48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8770e855ed898155 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.000+0000 7fda51ffb700 1 -- 172.21.15.22:0/399198787 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4800b8a0 con 0x7fda540b36f0 2023-07-06T01:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.000+0000 7fda51ffb700 1 -- 172.21.15.22:0/399198787 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4800ba00 con 0x7fda540b36f0 2023-07-06T01:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.000+0000 7fda51ffb700 1 -- 172.21.15.22:0/399198787 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4800bd90 con 0x7fda540b36f0 2023-07-06T01:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.000+0000 7fda59820700 1 -- 172.21.15.22:0/399198787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda540b36f0 msgr2=0x7fda540cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.000+0000 7fda59820700 1 --2- 172.21.15.22:0/399198787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda540b36f0 0x7fda540cf060 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fda480098f0 tx=0x7fda48009c00 comp rx=0 tx=0).stop 2023-07-06T01:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.000+0000 7fda59820700 1 -- 172.21.15.22:0/399198787 shutdown_connections 2023-07-06T01:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.000+0000 7fda59820700 1 --2- 172.21.15.22:0/399198787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda540b36f0 0x7fda540cf060 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:24.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.000+0000 7fda59820700 1 --2- 172.21.15.22:0/399198787 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda540cf5a0 0x7fda540d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.000+0000 7fda59820700 1 -- 172.21.15.22:0/399198787 >> 172.21.15.22:0/399198787 conn(0x7fda5409c2e0 msgr2=0x7fda540d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.000+0000 7fda59820700 1 -- 172.21.15.22:0/399198787 shutdown_connections 2023-07-06T01:20:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.000+0000 7fda59820700 1 -- 172.21.15.22:0/399198787 wait complete. 2023-07-06T01:20:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.001+0000 7fda59820700 1 Processor -- start 2023-07-06T01:20:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.002+0000 7fda59820700 1 -- start start 2023-07-06T01:20:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.002+0000 7fda59820700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda540b36f0 0x7fda541f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:24.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.002+0000 7fda59820700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda540b3b70 0x7fda541516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:24.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.002+0000 7fda59820700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda540cf5a0 0x7fda5415d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:24.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.002+0000 7fda59820700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda541f6ae0 con 0x7fda540b3b70 2023-07-06T01:20:24.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.002+0000 7fda59820700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda54151be0 con 0x7fda540b36f0 2023-07-06T01:20:24.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.002+0000 7fda59820700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda54152950 con 0x7fda540cf5a0 2023-07-06T01:20:24.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.002+0000 7fda527fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda540b3b70 0x7fda541516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.002+0000 7fda527fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda540b3b70 0x7fda541516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47304/0 (socket says 172.21.15.22:47304) 2023-07-06T01:20:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.002+0000 7fda527fc700 1 -- 172.21.15.22:0/3826309464 learned_addr learned my addr 172.21.15.22:0/3826309464 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.003+0000 7fda52ffd700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda540b36f0 0x7fda541f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.003+0000 7fda537fe700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda540cf5a0 0x7fda5415d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.003+0000 7fda527fc700 1 -- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda540cf5a0 msgr2=0x7fda5415d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.003+0000 7fda527fc700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda540cf5a0 0x7fda5415d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.003+0000 7fda527fc700 1 -- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda540b36f0 msgr2=0x7fda541f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.003+0000 7fda527fc700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda540b36f0 0x7fda541f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.003+0000 7fda527fc700 1 -- 172.21.15.22:0/3826309464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda48009550 con 0x7fda540b3b70 2023-07-06T01:20:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.003+0000 7fda527fc700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda540b3b70 0x7fda541516a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fda4400e840 tx=0x7fda4400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.003+0000 7fda3bfff700 1 -- 172.21.15.22:0/3826309464 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4400c020 con 0x7fda540b3b70 2023-07-06T01:20:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.003+0000 7fda3bfff700 1 -- 172.21.15.22:0/3826309464 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44004230 con 0x7fda540b3b70 2023-07-06T01:20:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.003+0000 7fda59820700 1 -- 172.21.15.22:0/3826309464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda54151dd0 con 0x7fda540b3b70 2023-07-06T01:20:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.003+0000 7fda3bfff700 1 -- 172.21.15.22:0/3826309464 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda440036a0 con 0x7fda540b3b70 2023-07-06T01:20:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.003+0000 7fda59820700 1 -- 172.21.15.22:0/3826309464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda54152390 con 0x7fda540b3b70 2023-07-06T01:20:24.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.004+0000 7fda59820700 1 -- 172.21.15.22:0/3826309464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda540b4840 con 0x7fda540b3b70 2023-07-06T01:20:24.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.005+0000 7fda3bfff700 1 -- 172.21.15.22:0/3826309464 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda44003800 con 0x7fda540b3b70 2023-07-06T01:20:24.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.005+0000 7fda3bfff700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fda4006ace0 0x7fda4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:24.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.005+0000 7fda3bfff700 1 -- 172.21.15.22:0/3826309464 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fda440890c0 con 0x7fda540b3b70 2023-07-06T01:20:24.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.005+0000 7fda52ffd700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fda4006ace0 0x7fda4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:24.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.006+0000 7fda52ffd700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fda4006ace0 0x7fda4006d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fda48009c40 tx=0x7fda48008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:24.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.010+0000 7fda3bfff700 1 -- 172.21.15.22:0/3826309464 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fda440541d0 con 0x7fda540b3b70 2023-07-06T01:20:24.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.256+0000 7fda59820700 1 -- 172.21.15.22:0/3826309464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fda540caad0 con 0x7fda540b3b70 2023-07-06T01:20:24.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.837+0000 7fda3bfff700 1 -- 172.21.15.22:0/3826309464 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7fda540caad0 con 0x7fda540b3b70 2023-07-06T01:20:24.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.841+0000 7fda59820700 1 -- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fda4006ace0 msgr2=0x7fda4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:24.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.841+0000 7fda59820700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fda4006ace0 0x7fda4006d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fda48009c40 tx=0x7fda48008030 comp rx=0 tx=0).stop 2023-07-06T01:20:24.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.841+0000 7fda59820700 1 -- 172.21.15.22:0/3826309464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda540b3b70 msgr2=0x7fda541516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:24.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.841+0000 7fda59820700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda540b3b70 0x7fda541516a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fda4400e840 tx=0x7fda4400eb50 comp rx=0 tx=0).stop 2023-07-06T01:20:24.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.842+0000 7fda59820700 1 -- 172.21.15.22:0/3826309464 shutdown_connections 2023-07-06T01:20:24.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.842+0000 7fda59820700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda540cf5a0 0x7fda5415d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:24.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.842+0000 7fda59820700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fda4006ace0 0x7fda4006d190 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fda48009c40 tx=0x7fda48008030 comp rx=0 tx=0).stop 2023-07-06T01:20:24.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.842+0000 7fda59820700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda540b36f0 0x7fda541f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:24.844 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.842+0000 7fda59820700 1 --2- 172.21.15.22:0/3826309464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda540b3b70 0x7fda541516a0 secure :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fda4400e840 tx=0x7fda4400eb50 comp rx=0 tx=0).stop 2023-07-06T01:20:24.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.842+0000 7fda59820700 1 -- 172.21.15.22:0/3826309464 >> 172.21.15.22:0/3826309464 conn(0x7fda5409c2e0 msgr2=0x7fda5406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:24.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.843+0000 7fda59820700 1 -- 172.21.15.22:0/3826309464 shutdown_connections 2023-07-06T01:20:24.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.847+0000 7fda59820700 1 -- 172.21.15.22:0/3826309464 wait complete. 2023-07-06T01:20:24.858 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 5 and data pool 6 2023-07-06T01:20:24.874 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:20:25.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.998+0000 7f82834c6700 1 Processor -- start 2023-07-06T01:20:25.000 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.998+0000 7f82834c6700 1 -- start start 2023-07-06T01:20:25.001 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.999+0000 7f82834c6700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f827c159360 0x7f827c15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.999+0000 7f82834c6700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f827c0cc660 0x7f827c0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.999+0000 7f82834c6700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827c0cf230 con 0x7f827c070a20 2023-07-06T01:20:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.999+0000 7f82834c6700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827c15df20 con 0x7f827c159360 2023-07-06T01:20:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.999+0000 7f82834c6700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827c0cd020 con 0x7f827c0cc660 2023-07-06T01:20:25.002 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.999+0000 7f8281262700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f827c159360 0x7f827c15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.999+0000 7f8281262700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f827c159360 0x7f827c15d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47326/0 (socket says 172.21.15.22:47326) 2023-07-06T01:20:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:24.999+0000 7f8281262700 1 -- 172.21.15.22:0/2931173446 learned_addr learned my addr 172.21.15.22:0/2931173446 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.000+0000 7f8280a61700 1 --2- 172.21.15.22:0/2931173446 >> v2:172.21.15.103:3300/0 conn(0x7f827c0cc660 0x7f827c0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.000+0000 7f8281262700 1 -- 172.21.15.22:0/2931173446 >> v2:172.21.15.103:3300/0 conn(0x7f827c0cc660 msgr2=0x7f827c0ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.000+0000 7f8281262700 1 --2- 172.21.15.22:0/2931173446 >> v2:172.21.15.103:3300/0 conn(0x7f827c0cc660 0x7f827c0ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.000+0000 7f8281262700 1 -- 172.21.15.22:0/2931173446 >> v1:172.21.15.22:6789/0 conn(0x7f827c070a20 legacy=0x7f827c070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.000+0000 7f8281262700 1 -- 172.21.15.22:0/2931173446 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c0ce620 con 0x7f827c159360 2023-07-06T01:20:25.003 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.000+0000 7f8281262700 1 --2- 172.21.15.22:0/2931173446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c159360 0x7f827c15d9e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f827c0af7d0 tx=0x7f826c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db16291bab848d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.000+0000 7f8273fff700 1 -- 172.21.15.22:0/2931173446 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c00dd30 con 0x7f827c159360 2023-07-06T01:20:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.000+0000 7f8273fff700 1 -- 172.21.15.22:0/2931173446 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826c00de90 con 0x7f827c159360 2023-07-06T01:20:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.000+0000 7f8273fff700 1 -- 172.21.15.22:0/2931173446 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c010690 con 0x7f827c159360 2023-07-06T01:20:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.001+0000 7f82834c6700 1 -- 172.21.15.22:0/2931173446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c159360 msgr2=0x7f827c15d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.001+0000 7f82834c6700 1 --2- 172.21.15.22:0/2931173446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c159360 0x7f827c15d9e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f827c0af7d0 tx=0x7f826c00be70 comp rx=0 tx=0).stop 2023-07-06T01:20:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.001+0000 7f82834c6700 1 -- 172.21.15.22:0/2931173446 shutdown_connections 2023-07-06T01:20:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.001+0000 7f82834c6700 1 --2- 172.21.15.22:0/2931173446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c159360 0x7f827c15d9e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.001+0000 7f82834c6700 1 --2- 172.21.15.22:0/2931173446 >> v2:172.21.15.103:3300/0 conn(0x7f827c0cc660 0x7f827c0ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.001+0000 7f82834c6700 1 -- 172.21.15.22:0/2931173446 >> 172.21.15.22:0/2931173446 conn(0x7f827c0b1890 msgr2=0x7f827c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.001+0000 7f82834c6700 1 -- 172.21.15.22:0/2931173446 shutdown_connections 2023-07-06T01:20:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.001+0000 7f82834c6700 1 -- 172.21.15.22:0/2931173446 wait complete. 2023-07-06T01:20:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.002+0000 7f82834c6700 1 Processor -- start 2023-07-06T01:20:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.002+0000 7f82834c6700 1 -- start start 2023-07-06T01:20:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.003+0000 7f82834c6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f827c070a20 0x7f827c0c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.003+0000 7f82834c6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f827c0cc660 0x7f827c06e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.003+0000 7f82834c6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c159360 0x7f827c0b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.003+0000 7f82834c6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c1534b0 con 0x7f827c159360 2023-07-06T01:20:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.003+0000 7f82834c6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c153790 con 0x7f827c0cc660 2023-07-06T01:20:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.003+0000 7f82834c6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c153a70 con 0x7f827c070a20 2023-07-06T01:20:25.010 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.003+0000 7f8281a63700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c159360 0x7f827c0b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.003+0000 7f8281a63700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c159360 0x7f827c0b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47336/0 (socket says 172.21.15.22:47336) 2023-07-06T01:20:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.003+0000 7f8281a63700 1 -- 172.21.15.22:0/3308679384 learned_addr learned my addr 172.21.15.22:0/3308679384 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f8281a63700 1 -- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f827c070a20 msgr2=0x7f827c0c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f8281262700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f827c070a20 0x7f827c0c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f8280a61700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f827c0cc660 0x7f827c06e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f8281a63700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f827c070a20 0x7f827c0c5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.011 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f8281a63700 1 -- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f827c0cc660 msgr2=0x7f827c06e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f8281a63700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f827c0cc660 0x7f827c06e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f8281a63700 1 -- 172.21.15.22:0/3308679384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c00b7f0 con 0x7f827c159360 2023-07-06T01:20:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f8280a61700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f827c0cc660 0x7f827c06e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f8281a63700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c159360 0x7f827c0b4290 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f827400cc50 tx=0x7f827400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f8271ffb700 1 -- 172.21.15.22:0/3308679384 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827400ad80 con 0x7f827c159360 2023-07-06T01:20:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f8271ffb700 1 -- 172.21.15.22:0/3308679384 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827400ce50 con 0x7f827c159360 2023-07-06T01:20:25.012 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f82834c6700 1 -- 172.21.15.22:0/3308679384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827c06df60 con 0x7f827c159360 2023-07-06T01:20:25.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f8271ffb700 1 -- 172.21.15.22:0/3308679384 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8274003c40 con 0x7f827c159360 2023-07-06T01:20:25.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.004+0000 7f82834c6700 1 -- 172.21.15.22:0/3308679384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827c0c5f40 con 0x7f827c159360 2023-07-06T01:20:25.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.005+0000 7f82834c6700 1 -- 172.21.15.22:0/3308679384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827c06e480 con 0x7f827c159360 2023-07-06T01:20:25.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.006+0000 7f8271ffb700 1 -- 172.21.15.22:0/3308679384 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f82740050d0 con 0x7f827c159360 2023-07-06T01:20:25.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.006+0000 7f8271ffb700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f826006ad10 0x7f826006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.006+0000 7f8271ffb700 1 -- 172.21.15.22:0/3308679384 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82740895e0 con 0x7f827c159360 2023-07-06T01:20:25.013 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.006+0000 7f8281262700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f826006ad10 0x7f826006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.007+0000 7f8281262700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f826006ad10 0x7f826006d1c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f826c000c00 tx=0x7f826c012000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.011+0000 7f8271ffb700 1 -- 172.21.15.22:0/3308679384 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f827c06e480 con 0x7f827c159360 2023-07-06T01:20:25.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.212+0000 7f82834c6700 1 -- 172.21.15.22:0/3308679384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f827c06e480 con 0x7f827c159360 2023-07-06T01:20:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.213+0000 7f8271ffb700 1 -- 172.21.15.22:0/3308679384 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+18294 (secure 0 0 0) 0x7f827c06e480 con 0x7f827c159360 2023-07-06T01:20:25.215 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:25.216 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:20:24.257669+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:20:22.043370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T01:20:23.225298+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:20:25.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.215+0000 7f82834c6700 1 -- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f826006ad10 msgr2=0x7f826006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.215+0000 7f82834c6700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f826006ad10 0x7f826006d1c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f826c000c00 tx=0x7f826c012000 comp rx=0 tx=0).stop 2023-07-06T01:20:25.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.215+0000 7f82834c6700 1 -- 172.21.15.22:0/3308679384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c159360 msgr2=0x7f827c0b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.215+0000 7f82834c6700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c159360 0x7f827c0b4290 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f827400cc50 tx=0x7f827400a340 comp rx=0 tx=0).stop 2023-07-06T01:20:25.218 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.215+0000 7f82834c6700 1 -- 172.21.15.22:0/3308679384 shutdown_connections 2023-07-06T01:20:25.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.215+0000 7f82834c6700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c159360 0x7f827c0b4290 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.215+0000 7f82834c6700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f827c070a20 0x7f827c0c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.215+0000 7f82834c6700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f826006ad10 0x7f826006d1c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.219 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.215+0000 7f82834c6700 1 --2- 172.21.15.22:0/3308679384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f827c0cc660 0x7f827c06e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.215+0000 7f82834c6700 1 -- 172.21.15.22:0/3308679384 >> 172.21.15.22:0/3308679384 conn(0x7f827c0b1890 msgr2=0x7f827c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.215+0000 7f82834c6700 1 -- 172.21.15.22:0/3308679384 shutdown_connections 2023-07-06T01:20:25.220 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.215+0000 7f82834c6700 1 -- 172.21.15.22:0/3308679384 wait complete. 2023-07-06T01:20:25.229 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:20:25.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.353+0000 7f249433b700 1 Processor -- start 2023-07-06T01:20:25.355 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.353+0000 7f249433b700 1 -- start start 2023-07-06T01:20:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.354+0000 7f249433b700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f248c15a710 0x7f248c0ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.354+0000 7f249433b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f248c0cef70 0x7f248c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.354+0000 7f249433b700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248c0af270 con 0x7f248c0aecb0 2023-07-06T01:20:25.356 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.354+0000 7f249433b700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248c0af550 con 0x7f248c15a710 2023-07-06T01:20:25.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.354+0000 7f249433b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0af830 con 0x7f248c0cef70 2023-07-06T01:20:25.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.354+0000 7f24928d8700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f248c0cef70 0x7f248c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.354+0000 7f24920d7700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f248c15a710 0x7f248c0ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.354+0000 7f24920d7700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f248c15a710 0x7f248c0ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:36064/0 (socket says 172.21.15.22:36064) 2023-07-06T01:20:25.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.354+0000 7f24928d8700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f248c0cef70 0x7f248c0cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:58482/0 (socket says 172.21.15.22:58482) 2023-07-06T01:20:25.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.354+0000 7f24920d7700 1 -- 172.21.15.22:0/4285318279 learned_addr learned my addr 172.21.15.22:0/4285318279 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:25.357 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.354+0000 7f24918d6700 1 --1- 172.21.15.22:0/4285318279 >> v1:172.21.15.103:6789/0 conn(0x7f248c0aecb0 0x7f248c0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:40150/0 (socket says 172.21.15.22:40150) 2023-07-06T01:20:25.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.355+0000 7f24928d8700 1 -- 172.21.15.22:0/4285318279 >> v2:172.21.15.103:3300/0 conn(0x7f248c15a710 msgr2=0x7f248c0ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:25.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.355+0000 7f24928d8700 1 --2- 172.21.15.22:0/4285318279 >> v2:172.21.15.103:3300/0 conn(0x7f248c15a710 0x7f248c0ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.355+0000 7f24928d8700 1 -- 172.21.15.22:0/4285318279 >> v1:172.21.15.103:6789/0 conn(0x7f248c0aecb0 legacy=0x7f248c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.355+0000 7f24928d8700 1 -- 172.21.15.22:0/4285318279 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c0cb2e0 con 0x7f248c0cef70 2023-07-06T01:20:25.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.355+0000 7f24920d7700 1 --2- 172.21.15.22:0/4285318279 >> v2:172.21.15.103:3300/0 conn(0x7f248c15a710 0x7f248c0ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:25.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.355+0000 7f24928d8700 1 --2- 172.21.15.22:0/4285318279 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f248c0cef70 0x7f248c0cbae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2484009d90 tx=0x7f248400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30123d023a7e5d2f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:25.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.355+0000 7f24910d5700 1 -- 172.21.15.22:0/4285318279 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248400e070 con 0x7f248c0cef70 2023-07-06T01:20:25.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.355+0000 7f24910d5700 1 -- 172.21.15.22:0/4285318279 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248400bb20 con 0x7f248c0cef70 2023-07-06T01:20:25.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.355+0000 7f24910d5700 1 -- 172.21.15.22:0/4285318279 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484012430 con 0x7f248c0cef70 2023-07-06T01:20:25.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.356+0000 7f249433b700 1 -- 172.21.15.22:0/4285318279 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f248c0cef70 msgr2=0x7f248c0cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.356+0000 7f249433b700 1 --2- 172.21.15.22:0/4285318279 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f248c0cef70 0x7f248c0cbae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2484009d90 tx=0x7f248400b330 comp rx=0 tx=0).stop 2023-07-06T01:20:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.356+0000 7f249433b700 1 -- 172.21.15.22:0/4285318279 shutdown_connections 2023-07-06T01:20:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.356+0000 7f249433b700 1 --2- 172.21.15.22:0/4285318279 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f248c0cef70 0x7f248c0cbae0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.356+0000 7f249433b700 1 --2- 172.21.15.22:0/4285318279 >> v2:172.21.15.103:3300/0 conn(0x7f248c15a710 0x7f248c0ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.356+0000 7f249433b700 1 -- 172.21.15.22:0/4285318279 >> 172.21.15.22:0/4285318279 conn(0x7f248c0999f0 msgr2=0x7f248c099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.356+0000 7f249433b700 1 -- 172.21.15.22:0/4285318279 shutdown_connections 2023-07-06T01:20:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.356+0000 7f249433b700 1 -- 172.21.15.22:0/4285318279 wait complete. 2023-07-06T01:20:25.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.357+0000 7f249433b700 1 Processor -- start 2023-07-06T01:20:25.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.357+0000 7f249433b700 1 -- start start 2023-07-06T01:20:25.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f249433b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0aecb0 0x7f248c1f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f249433b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f248c0cef70 0x7f248c1f0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f249433b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f248c15a710 0x7f248c1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f249433b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0c6140 con 0x7f248c0aecb0 2023-07-06T01:20:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f249433b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0b1da0 con 0x7f248c0cef70 2023-07-06T01:20:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f249433b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0c5810 con 0x7f248c15a710 2023-07-06T01:20:25.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f24920d7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0aecb0 0x7f248c1f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f24920d7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0aecb0 0x7f248c1f10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47346/0 (socket says 172.21.15.22:47346) 2023-07-06T01:20:25.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f24920d7700 1 -- 172.21.15.22:0/1766037168 learned_addr learned my addr 172.21.15.22:0/1766037168 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:25.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f24928d8700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f248c15a710 0x7f248c1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f24918d6700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f248c0cef70 0x7f248c1f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f24920d7700 1 -- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f248c15a710 msgr2=0x7f248c1569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f24920d7700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f248c15a710 0x7f248c1569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f24920d7700 1 -- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f248c0cef70 msgr2=0x7f248c1f0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f24920d7700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f248c0cef70 0x7f248c1f0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f24920d7700 1 -- 172.21.15.22:0/1766037168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c009550 con 0x7f248c0aecb0 2023-07-06T01:20:25.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.358+0000 7f24920d7700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0aecb0 0x7f248c1f10f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f247c00e9b0 tx=0x7f247c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:25.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.359+0000 7f2482ffd700 1 -- 172.21.15.22:0/1766037168 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00ee50 con 0x7f248c0aecb0 2023-07-06T01:20:25.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.359+0000 7f2482ffd700 1 -- 172.21.15.22:0/1766037168 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c00c850 con 0x7f248c0aecb0 2023-07-06T01:20:25.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.359+0000 7f249433b700 1 -- 172.21.15.22:0/1766037168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2484009a70 con 0x7f248c0aecb0 2023-07-06T01:20:25.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.359+0000 7f2482ffd700 1 -- 172.21.15.22:0/1766037168 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00cbc0 con 0x7f248c0aecb0 2023-07-06T01:20:25.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.359+0000 7f249433b700 1 -- 172.21.15.22:0/1766037168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248c0c5dd0 con 0x7f248c0aecb0 2023-07-06T01:20:25.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.360+0000 7f249433b700 1 -- 172.21.15.22:0/1766037168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248c0744b0 con 0x7f248c0aecb0 2023-07-06T01:20:25.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.360+0000 7f2482ffd700 1 -- 172.21.15.22:0/1766037168 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f247c00cd20 con 0x7f248c0aecb0 2023-07-06T01:20:25.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.361+0000 7f2482ffd700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f247006ace0 0x7f247006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.361+0000 7f2482ffd700 1 -- 172.21.15.22:0/1766037168 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f247c089430 con 0x7f248c0aecb0 2023-07-06T01:20:25.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.361+0000 7f24918d6700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f247006ace0 0x7f247006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.361+0000 7f24918d6700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f247006ace0 0x7f247006d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2488000f60 tx=0x7f2488007ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:25.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.365+0000 7f2482ffd700 1 -- 172.21.15.22:0/1766037168 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f247c054570 con 0x7f248c0aecb0 2023-07-06T01:20:25.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.568+0000 7f249433b700 1 -- 172.21.15.22:0/1766037168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f248c0cfcb0 con 0x7f248c0aecb0 2023-07-06T01:20:25.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.569+0000 7f2482ffd700 1 -- 172.21.15.22:0/1766037168 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+18294 (secure 0 0 0) 0x7f247c057b90 con 0x7f248c0aecb0 2023-07-06T01:20:25.571 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:25.571 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:20:24.257669+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:20:22.043370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T01:20:23.225298+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:20:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.571+0000 7f249433b700 1 -- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f247006ace0 msgr2=0x7f247006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.572+0000 7f249433b700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f247006ace0 0x7f247006d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2488000f60 tx=0x7f2488007ed0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.572+0000 7f249433b700 1 -- 172.21.15.22:0/1766037168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0aecb0 msgr2=0x7f248c1f10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.572+0000 7f249433b700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0aecb0 0x7f248c1f10f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f247c00e9b0 tx=0x7f247c00ecc0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.572+0000 7f249433b700 1 -- 172.21.15.22:0/1766037168 shutdown_connections 2023-07-06T01:20:25.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.572+0000 7f249433b700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f248c15a710 0x7f248c1569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.572+0000 7f249433b700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0aecb0 0x7f248c1f10f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.572+0000 7f249433b700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f247006ace0 0x7f247006d190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.572+0000 7f249433b700 1 --2- 172.21.15.22:0/1766037168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f248c0cef70 0x7f248c1f0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.572+0000 7f249433b700 1 -- 172.21.15.22:0/1766037168 >> 172.21.15.22:0/1766037168 conn(0x7f248c0999f0 msgr2=0x7f248c0ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:25.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.572+0000 7f249433b700 1 -- 172.21.15.22:0/1766037168 shutdown_connections 2023-07-06T01:20:25.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.572+0000 7f249433b700 1 -- 172.21.15.22:0/1766037168 wait complete. 2023-07-06T01:20:25.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T01:20:25.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.712+0000 7fcfcb332700 1 Processor -- start 2023-07-06T01:20:25.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.712+0000 7fcfcb332700 1 -- start start 2023-07-06T01:20:25.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.713+0000 7fcfcb332700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcfc40ccd70 0x7fcfc415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.713+0000 7fcfcb332700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcfc406d040 0x7fcfc406d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.713+0000 7fcfcb332700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc40cf230 con 0x7fcfc4159360 2023-07-06T01:20:25.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.713+0000 7fcfcb332700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc415df20 con 0x7fcfc40ccd70 2023-07-06T01:20:25.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.713+0000 7fcfcb332700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc40ce620 con 0x7fcfc406d040 2023-07-06T01:20:25.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.713+0000 7fcfc88cd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcfc40ccd70 0x7fcfc415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.713+0000 7fcfc88cd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcfc40ccd70 0x7fcfc415d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47356/0 (socket says 172.21.15.22:47356) 2023-07-06T01:20:25.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.713+0000 7fcfc88cd700 1 -- 172.21.15.22:0/1496475866 learned_addr learned my addr 172.21.15.22:0/1496475866 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:25.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.714+0000 7fcfc98cf700 1 --2- 172.21.15.22:0/1496475866 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcfc406d040 0x7fcfc406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.714+0000 7fcfc88cd700 1 -- 172.21.15.22:0/1496475866 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcfc406d040 msgr2=0x7fcfc406d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:25.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.714+0000 7fcfc88cd700 1 --2- 172.21.15.22:0/1496475866 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcfc406d040 0x7fcfc406d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.714+0000 7fcfc88cd700 1 -- 172.21.15.22:0/1496475866 >> v1:172.21.15.103:6789/0 conn(0x7fcfc4159360 legacy=0x7fcfc40cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.714+0000 7fcfc88cd700 1 -- 172.21.15.22:0/1496475866 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc40ce900 con 0x7fcfc40ccd70 2023-07-06T01:20:25.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.714+0000 7fcfc88cd700 1 --2- 172.21.15.22:0/1496475866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc40ccd70 0x7fcfc415d9e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00098f0 tx=0x7fcfc0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e29f0ce0e5a5c52 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:25.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.714+0000 7fcfbbfff700 1 -- 172.21.15.22:0/1496475866 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc000b8a0 con 0x7fcfc40ccd70 2023-07-06T01:20:25.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.714+0000 7fcfbbfff700 1 -- 172.21.15.22:0/1496475866 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc000ba00 con 0x7fcfc40ccd70 2023-07-06T01:20:25.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.714+0000 7fcfbbfff700 1 -- 172.21.15.22:0/1496475866 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc000bd90 con 0x7fcfc40ccd70 2023-07-06T01:20:25.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.714+0000 7fcfcb332700 1 -- 172.21.15.22:0/1496475866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc40ccd70 msgr2=0x7fcfc415d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.714+0000 7fcfcb332700 1 --2- 172.21.15.22:0/1496475866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc40ccd70 0x7fcfc415d9e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00098f0 tx=0x7fcfc0009c00 comp rx=0 tx=0).stop 2023-07-06T01:20:25.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.715+0000 7fcfcb332700 1 -- 172.21.15.22:0/1496475866 shutdown_connections 2023-07-06T01:20:25.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.715+0000 7fcfcb332700 1 --2- 172.21.15.22:0/1496475866 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcfc406d040 0x7fcfc406d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.715+0000 7fcfcb332700 1 --2- 172.21.15.22:0/1496475866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc40ccd70 0x7fcfc415d9e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.715+0000 7fcfcb332700 1 -- 172.21.15.22:0/1496475866 >> 172.21.15.22:0/1496475866 conn(0x7fcfc40b1890 msgr2=0x7fcfc40b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:25.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.715+0000 7fcfcb332700 1 -- 172.21.15.22:0/1496475866 shutdown_connections 2023-07-06T01:20:25.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.715+0000 7fcfcb332700 1 -- 172.21.15.22:0/1496475866 wait complete. 2023-07-06T01:20:25.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.716+0000 7fcfcb332700 1 Processor -- start 2023-07-06T01:20:25.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.716+0000 7fcfcb332700 1 -- start start 2023-07-06T01:20:25.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfcb332700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc406d040 0x7fcfc415a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfcb332700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcfc40ccd70 0x7fcfc40c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfcb332700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfc4159360 0x7fcfc40c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfcb332700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc415bd10 con 0x7fcfc406d040 2023-07-06T01:20:25.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfcb332700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc415b100 con 0x7fcfc4159360 2023-07-06T01:20:25.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfcb332700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc415b3e0 con 0x7fcfc40ccd70 2023-07-06T01:20:25.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfc90ce700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc406d040 0x7fcfc415a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfc90ce700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc406d040 0x7fcfc415a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47360/0 (socket says 172.21.15.22:47360) 2023-07-06T01:20:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfc90ce700 1 -- 172.21.15.22:0/3429067779 learned_addr learned my addr 172.21.15.22:0/3429067779 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfc98cf700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfc4159360 0x7fcfc40c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfc88cd700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcfc40ccd70 0x7fcfc40c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfc90ce700 1 -- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcfc40ccd70 msgr2=0x7fcfc40c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfc90ce700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcfc40ccd70 0x7fcfc40c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfc90ce700 1 -- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfc4159360 msgr2=0x7fcfc40c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfc90ce700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfc4159360 0x7fcfc40c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfc90ce700 1 -- 172.21.15.22:0/3429067779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc0009550 con 0x7fcfc406d040 2023-07-06T01:20:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfc90ce700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc406d040 0x7fcfc415a8b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400c450 tx=0x7fcfb400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfb9ffb700 1 -- 172.21.15.22:0/3429067779 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40052d0 con 0x7fcfc406d040 2023-07-06T01:20:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.717+0000 7fcfb9ffb700 1 -- 172.21.15.22:0/3429067779 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb4005430 con 0x7fcfc406d040 2023-07-06T01:20:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.718+0000 7fcfcb332700 1 -- 172.21.15.22:0/3429067779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc415b6c0 con 0x7fcfc406d040 2023-07-06T01:20:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.718+0000 7fcfb9ffb700 1 -- 172.21.15.22:0/3429067779 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40038c0 con 0x7fcfc406d040 2023-07-06T01:20:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.718+0000 7fcfcb332700 1 -- 172.21.15.22:0/3429067779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc415c1a0 con 0x7fcfc406d040 2023-07-06T01:20:25.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.719+0000 7fcfcb332700 1 -- 172.21.15.22:0/3429067779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc40744b0 con 0x7fcfc406d040 2023-07-06T01:20:25.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.719+0000 7fcfb9ffb700 1 -- 172.21.15.22:0/3429067779 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcfb4003a20 con 0x7fcfc406d040 2023-07-06T01:20:25.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.719+0000 7fcfb9ffb700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fcfa806ace0 0x7fcfa806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:25.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.719+0000 7fcfb9ffb700 1 -- 172.21.15.22:0/3429067779 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcfb4088880 con 0x7fcfc406d040 2023-07-06T01:20:25.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.720+0000 7fcfc88cd700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fcfa806ace0 0x7fcfa806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:25.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.720+0000 7fcfc88cd700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fcfa806ace0 0x7fcfa806d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40cd1b0 tx=0x7fcfc0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:25.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.724+0000 7fcfb9ffb700 1 -- 172.21.15.22:0/3429067779 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcfb4053890 con 0x7fcfc406d040 2023-07-06T01:20:25.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:25.977+0000 7fcfcb332700 1 -- 172.21.15.22:0/3429067779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcfc415bff0 con 0x7fcfc406d040 2023-07-06T01:20:26.718 INFO:tasks.ceph.mds.c.smithi022.stderr:2023-07-06T01:20:26.717+0000 7f256ef73700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T01:20:26.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:26.866+0000 7fcfb9ffb700 1 -- 172.21.15.22:0/3429067779 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcfb400ce60 con 0x7fcfc406d040 2023-07-06T01:20:26.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:26.869+0000 7fcfcb332700 1 -- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fcfa806ace0 msgr2=0x7fcfa806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:26.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:26.869+0000 7fcfcb332700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fcfa806ace0 0x7fcfa806d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40cd1b0 tx=0x7fcfc0008030 comp rx=0 tx=0).stop 2023-07-06T01:20:26.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:26.869+0000 7fcfcb332700 1 -- 172.21.15.22:0/3429067779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc406d040 msgr2=0x7fcfc415a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:26.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:26.869+0000 7fcfcb332700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc406d040 0x7fcfc415a8b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400c450 tx=0x7fcfb400c760 comp rx=0 tx=0).stop 2023-07-06T01:20:26.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:26.869+0000 7fcfcb332700 1 -- 172.21.15.22:0/3429067779 shutdown_connections 2023-07-06T01:20:26.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:26.869+0000 7fcfcb332700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fcfa806ace0 0x7fcfa806d190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:26.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:26.869+0000 7fcfcb332700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfc4159360 0x7fcfc40c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:26.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:26.869+0000 7fcfcb332700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcfc406d040 0x7fcfc415a8b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:26.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:26.869+0000 7fcfcb332700 1 --2- 172.21.15.22:0/3429067779 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcfc40ccd70 0x7fcfc40c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:26.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:26.869+0000 7fcfcb332700 1 -- 172.21.15.22:0/3429067779 >> 172.21.15.22:0/3429067779 conn(0x7fcfc40b1890 msgr2=0x7fcfc406ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:26.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:26.869+0000 7fcfcb332700 1 -- 172.21.15.22:0/3429067779 shutdown_connections 2023-07-06T01:20:26.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:26.869+0000 7fcfcb332700 1 -- 172.21.15.22:0/3429067779 wait complete. 2023-07-06T01:20:26.886 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T01:20:26.886 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T01:20:27.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.019+0000 7fbbf2be9700 1 Processor -- start 2023-07-06T01:20:27.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.019+0000 7fbbf2be9700 1 -- start start 2023-07-06T01:20:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.020+0000 7fbbf2be9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbbec15d9e0 0x7fbbec06d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.020+0000 7fbbf2be9700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbec0cf230 con 0x7fbbec159360 2023-07-06T01:20:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.020+0000 7fbbf2be9700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbec0ce620 con 0x7fbbec0ccd70 2023-07-06T01:20:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.020+0000 7fbbf2be9700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbec0ce900 con 0x7fbbec15d9e0 2023-07-06T01:20:27.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.020+0000 7fbbf1186700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbbec15d9e0 0x7fbbec06d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.020+0000 7fbbf0985700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fbbec159360 0x7fbbec0cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38112/0 (socket says 172.21.15.22:38112) 2023-07-06T01:20:27.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.020+0000 7fbbf0985700 1 -- 172.21.15.22:0/1291088886 learned_addr learned my addr 172.21.15.22:0/1291088886 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.021+0000 7fbbf1186700 1 -- 172.21.15.22:0/1291088886 >> v1:172.21.15.103:6789/0 conn(0x7fbbec0ccd70 legacy=0x7fbbec06d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.021+0000 7fbbf1186700 1 -- 172.21.15.22:0/1291088886 >> v1:172.21.15.22:6789/0 conn(0x7fbbec159360 legacy=0x7fbbec0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.021+0000 7fbbf1186700 1 -- 172.21.15.22:0/1291088886 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec0cebe0 con 0x7fbbec15d9e0 2023-07-06T01:20:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.021+0000 7fbbf1186700 1 --2- 172.21.15.22:0/1291088886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbec15d9e0 0x7fbbec06d750 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009d90 tx=0x7fbbe400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b874cbbf188e24 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:27.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.021+0000 7fbbeb7fe700 1 -- 172.21.15.22:0/1291088886 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe400bdd0 con 0x7fbbec15d9e0 2023-07-06T01:20:27.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.021+0000 7fbbeb7fe700 1 -- 172.21.15.22:0/1291088886 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe4011460 con 0x7fbbec15d9e0 2023-07-06T01:20:27.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.021+0000 7fbbeb7fe700 1 -- 172.21.15.22:0/1291088886 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4011810 con 0x7fbbec15d9e0 2023-07-06T01:20:27.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.021+0000 7fbbf2be9700 1 -- 172.21.15.22:0/1291088886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbec15d9e0 msgr2=0x7fbbec06d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:27.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.021+0000 7fbbf2be9700 1 --2- 172.21.15.22:0/1291088886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbec15d9e0 0x7fbbec06d750 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009d90 tx=0x7fbbe400b330 comp rx=0 tx=0).stop 2023-07-06T01:20:27.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.022+0000 7fbbf2be9700 1 -- 172.21.15.22:0/1291088886 shutdown_connections 2023-07-06T01:20:27.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.022+0000 7fbbf2be9700 1 --2- 172.21.15.22:0/1291088886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbec15d9e0 0x7fbbec06d750 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:27.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.022+0000 7fbbf2be9700 1 -- 172.21.15.22:0/1291088886 >> 172.21.15.22:0/1291088886 conn(0x7fbbec0b1890 msgr2=0x7fbbec0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:27.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.022+0000 7fbbf2be9700 1 -- 172.21.15.22:0/1291088886 shutdown_connections 2023-07-06T01:20:27.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.022+0000 7fbbf2be9700 1 -- 172.21.15.22:0/1291088886 wait complete. 2023-07-06T01:20:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.023+0000 7fbbf2be9700 1 Processor -- start 2023-07-06T01:20:27.027 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.023+0000 7fbbf2be9700 1 -- start start 2023-07-06T01:20:27.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.024+0000 7fbbf2be9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbec0ccd70 0x7fbbec0b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:27.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.024+0000 7fbbf2be9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbec159360 0x7fbbec14bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:27.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.024+0000 7fbbf2be9700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbbec15d9e0 0x7fbbec14c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:27.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.024+0000 7fbbf2be9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec0c7a80 con 0x7fbbec0ccd70 2023-07-06T01:20:27.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.024+0000 7fbbf2be9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec0c71e0 con 0x7fbbec159360 2023-07-06T01:20:27.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.024+0000 7fbbf2be9700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec0c74c0 con 0x7fbbec15d9e0 2023-07-06T01:20:27.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.024+0000 7fbbf0985700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbec0ccd70 0x7fbbec0b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.024+0000 7fbbf0985700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbec0ccd70 0x7fbbec0b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47374/0 (socket says 172.21.15.22:47374) 2023-07-06T01:20:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.024+0000 7fbbf0985700 1 -- 172.21.15.22:0/2383668233 learned_addr learned my addr 172.21.15.22:0/2383668233 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.025+0000 7fbbebfff700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbec159360 0x7fbbec14bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.025+0000 7fbbf1186700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbbec15d9e0 0x7fbbec14c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.025+0000 7fbbf0985700 1 -- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbbec15d9e0 msgr2=0x7fbbec14c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.025+0000 7fbbf0985700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbbec15d9e0 0x7fbbec14c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.025+0000 7fbbf0985700 1 -- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbec159360 msgr2=0x7fbbec14bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:27.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.025+0000 7fbbf0985700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbec159360 0x7fbbec14bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:27.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.025+0000 7fbbf0985700 1 -- 172.21.15.22:0/2383668233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe4009a70 con 0x7fbbec0ccd70 2023-07-06T01:20:27.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.025+0000 7fbbf0985700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbec0ccd70 0x7fbbec0b51c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c520 tx=0x7fbbe000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:27.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.025+0000 7fbbe97fa700 1 -- 172.21.15.22:0/2383668233 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0005d70 con 0x7fbbec0ccd70 2023-07-06T01:20:27.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.025+0000 7fbbe97fa700 1 -- 172.21.15.22:0/2383668233 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe00070d0 con 0x7fbbec0ccd70 2023-07-06T01:20:27.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.025+0000 7fbbf2be9700 1 -- 172.21.15.22:0/2383668233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbec0c77a0 con 0x7fbbec0ccd70 2023-07-06T01:20:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.025+0000 7fbbe97fa700 1 -- 172.21.15.22:0/2383668233 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0007440 con 0x7fbbec0ccd70 2023-07-06T01:20:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.025+0000 7fbbf2be9700 1 -- 172.21.15.22:0/2383668233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbec0af740 con 0x7fbbec0ccd70 2023-07-06T01:20:27.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.026+0000 7fbbf2be9700 1 -- 172.21.15.22:0/2383668233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbec06d000 con 0x7fbbec0ccd70 2023-07-06T01:20:27.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.027+0000 7fbbe97fa700 1 -- 172.21.15.22:0/2383668233 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbbe00075a0 con 0x7fbbec0ccd70 2023-07-06T01:20:27.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.027+0000 7fbbe97fa700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbbd006ace0 0x7fbbd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:27.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.027+0000 7fbbe97fa700 1 -- 172.21.15.22:0/2383668233 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbbe0088650 con 0x7fbbec0ccd70 2023-07-06T01:20:27.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.027+0000 7fbbebfff700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbbd006ace0 0x7fbbd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:27.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.028+0000 7fbbebfff700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbbd006ace0 0x7fbbd006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8000f60 tx=0x7fbbd8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:27.033 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.032+0000 7fbbe97fa700 1 -- 172.21.15.22:0/2383668233 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbbec06d000 con 0x7fbbec0ccd70 2023-07-06T01:20:27.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.276+0000 7fbbf2be9700 1 -- 172.21.15.22:0/2383668233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbbec0af740 con 0x7fbbec0ccd70 2023-07-06T01:20:27.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.870+0000 7fbbe97fa700 1 -- 172.21.15.22:0/2383668233 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbbec0af740 con 0x7fbbec0ccd70 2023-07-06T01:20:27.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.873+0000 7fbbf2be9700 1 -- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbbd006ace0 msgr2=0x7fbbd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:27.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.873+0000 7fbbf2be9700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbbd006ace0 0x7fbbd006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8000f60 tx=0x7fbbd8009040 comp rx=0 tx=0).stop 2023-07-06T01:20:27.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.873+0000 7fbbf2be9700 1 -- 172.21.15.22:0/2383668233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbec0ccd70 msgr2=0x7fbbec0b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:27.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.873+0000 7fbbf2be9700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbec0ccd70 0x7fbbec0b51c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c520 tx=0x7fbbe000c8e0 comp rx=0 tx=0).stop 2023-07-06T01:20:27.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.873+0000 7fbbf2be9700 1 -- 172.21.15.22:0/2383668233 shutdown_connections 2023-07-06T01:20:27.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.873+0000 7fbbf2be9700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbbec15d9e0 0x7fbbec14c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:27.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.873+0000 7fbbf2be9700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbbec0ccd70 0x7fbbec0b51c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:27.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.873+0000 7fbbf2be9700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbbd006ace0 0x7fbbd006d190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:27.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.873+0000 7fbbf2be9700 1 --2- 172.21.15.22:0/2383668233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbbec159360 0x7fbbec14bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:27.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.873+0000 7fbbf2be9700 1 -- 172.21.15.22:0/2383668233 >> 172.21.15.22:0/2383668233 conn(0x7fbbec0b1890 msgr2=0x7fbbec155ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:27.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.873+0000 7fbbf2be9700 1 -- 172.21.15.22:0/2383668233 shutdown_connections 2023-07-06T01:20:27.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:27.873+0000 7fbbf2be9700 1 -- 172.21.15.22:0/2383668233 wait complete. 2023-07-06T01:20:27.893 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T01:20:27.893 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:20:28.025 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.024+0000 7f30cdde2700 1 Processor -- start 2023-07-06T01:20:28.026 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.024+0000 7f30cdde2700 1 -- start start 2023-07-06T01:20:28.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.025+0000 7f30cdde2700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30c815a6c0 0x7f30c815eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.025+0000 7f30cdde2700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c815f0b0 con 0x7f30c806a710 2023-07-06T01:20:28.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.025+0000 7f30cdde2700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c815aa90 con 0x7f30c8071980 2023-07-06T01:20:28.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.025+0000 7f30cdde2700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c806f260 con 0x7f30c815a6c0 2023-07-06T01:20:28.028 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.025+0000 7f30c7fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f30c806a710 0x7f30c80cbdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38130/0 (socket says 172.21.15.22:38130) 2023-07-06T01:20:28.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.025+0000 7f30c7fff700 1 -- 172.21.15.22:0/3973279476 learned_addr learned my addr 172.21.15.22:0/3973279476 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:28.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.025+0000 7f30c77fe700 1 --2- 172.21.15.22:0/3973279476 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30c815a6c0 0x7f30c815eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.025+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429361134 0 0) 0x7f30c0001490 con 0x7f30c806a710 2023-07-06T01:20:28.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.025+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30b40035b0 con 0x7f30c806a710 2023-07-06T01:20:28.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.025+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1273648589 0 0) 0x7f30b40035b0 con 0x7f30c806a710 2023-07-06T01:20:28.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.025+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30c815a6c0 msgr2=0x7f30c815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:28.029 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.025+0000 7f30c67fc700 1 --2- 172.21.15.22:0/3973279476 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30c815a6c0 0x7f30c815eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.025+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 >> v1:172.21.15.103:6789/0 conn(0x7f30c8071980 legacy=0x7f30c806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c806e5e0 con 0x7f30c806a710 2023-07-06T01:20:28.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f30c0001b80 con 0x7f30c806a710 2023-07-06T01:20:28.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 >> v1:172.21.15.22:6789/0 conn(0x7f30c806a710 legacy=0x7f30c80cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c67fc700 1 --2- 172.21.15.22:0/3973279476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30b4003a30 0x7f30b4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c67fc700 1 --2- 172.21.15.22:0/3973279476 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30c815a6c0 0x7f30b4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c67fc700 1 --2- 172.21.15.22:0/3973279476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30b4004d90 0x7f30b4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.030 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c806f260 con 0x7f30b4003a30 2023-07-06T01:20:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b400bbd0 con 0x7f30b4004d90 2023-07-06T01:20:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b400bd90 con 0x7f30c815a6c0 2023-07-06T01:20:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c77fe700 1 --2- 172.21.15.22:0/3973279476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30b4003a30 0x7f30b4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c6ffd700 1 --2- 172.21.15.22:0/3973279476 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30c815a6c0 0x7f30b4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c7fff700 1 --2- 172.21.15.22:0/3973279476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30b4004d90 0x7f30b4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c77fe700 1 -- 172.21.15.22:0/3973279476 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30c815a6c0 msgr2=0x7f30b4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c77fe700 1 --2- 172.21.15.22:0/3973279476 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30c815a6c0 0x7f30b4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.031 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c77fe700 1 -- 172.21.15.22:0/3973279476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30b4004d90 msgr2=0x7f30b4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c77fe700 1 --2- 172.21.15.22:0/3973279476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30b4004d90 0x7f30b4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c77fe700 1 -- 172.21.15.22:0/3973279476 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b40035b0 con 0x7f30b4003a30 2023-07-06T01:20:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c7fff700 1 --2- 172.21.15.22:0/3973279476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30b4004d90 0x7f30b4005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c77fe700 1 --2- 172.21.15.22:0/3973279476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30b4003a30 0x7f30b4003e40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009b30 tx=0x7f30b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7670ac6be524c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b800e6c0 con 0x7f30b4003a30 2023-07-06T01:20:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b800e820 con 0x7f30b4003a30 2023-07-06T01:20:28.032 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.026+0000 7f30c67fc700 1 -- 172.21.15.22:0/3973279476 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b800eb90 con 0x7f30b4003a30 2023-07-06T01:20:28.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.027+0000 7f30cdde2700 1 -- 172.21.15.22:0/3973279476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30b4003a30 msgr2=0x7f30b4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.027+0000 7f30cdde2700 1 --2- 172.21.15.22:0/3973279476 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30b4003a30 0x7f30b4003e40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f30b8009b30 tx=0x7f30b4005880 comp rx=0 tx=0).stop 2023-07-06T01:20:28.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.027+0000 7f30c7fff700 1 -- 172.21.15.22:0/3973279476 reap_dead start 2023-07-06T01:20:28.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.027+0000 7f30cdde2700 1 -- 172.21.15.22:0/3973279476 shutdown_connections 2023-07-06T01:20:28.035 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.027+0000 7f30cdde2700 1 -- 172.21.15.22:0/3973279476 >> 172.21.15.22:0/3973279476 conn(0x7f30c80b1890 msgr2=0x7f30c80b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.027+0000 7f30cdde2700 1 -- 172.21.15.22:0/3973279476 shutdown_connections 2023-07-06T01:20:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.027+0000 7f30cdde2700 1 -- 172.21.15.22:0/3973279476 wait complete. 2023-07-06T01:20:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.028+0000 7f30cdde2700 1 Processor -- start 2023-07-06T01:20:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.028+0000 7f30cdde2700 1 -- start start 2023-07-06T01:20:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.029+0000 7f30cdde2700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30b4003a30 0x7f30c81512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.029+0000 7f30cdde2700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30c8151c00 0x7f30c80c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.029+0000 7f30cdde2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30c80c9400 0x7f30c80c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.036 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.029+0000 7f30cdde2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80c8fe0 con 0x7f30c80c9400 2023-07-06T01:20:28.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.029+0000 7f30cdde2700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80c8360 con 0x7f30b4003a30 2023-07-06T01:20:28.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.029+0000 7f30cdde2700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80c8640 con 0x7f30c8151c00 2023-07-06T01:20:28.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.029+0000 7f30c7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30c80c9400 0x7f30c80c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.029+0000 7f30c7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30c80c9400 0x7f30c80c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47398/0 (socket says 172.21.15.22:47398) 2023-07-06T01:20:28.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.029+0000 7f30c7fff700 1 -- 172.21.15.22:0/2839476428 learned_addr learned my addr 172.21.15.22:0/2839476428 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:28.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.029+0000 7f30c77fe700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30b4003a30 0x7f30c81512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.029+0000 7f30c6ffd700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30c8151c00 0x7f30c80c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.037 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.029+0000 7f30c7fff700 1 -- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30c8151c00 msgr2=0x7f30c80c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.029+0000 7f30c7fff700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30c8151c00 0x7f30c80c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.030+0000 7f30c7fff700 1 -- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30b4003a30 msgr2=0x7f30c81512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.030+0000 7f30c7fff700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30b4003a30 0x7f30c81512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.030+0000 7f30c7fff700 1 -- 172.21.15.22:0/2839476428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c80c8920 con 0x7f30c80c9400 2023-07-06T01:20:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.030+0000 7f30c7fff700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30c80c9400 0x7f30c80c9880 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f30c80ca280 tx=0x7f30c000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.030+0000 7f30bffff700 1 -- 172.21.15.22:0/2839476428 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0001b70 con 0x7f30c80c9400 2023-07-06T01:20:28.038 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.030+0000 7f30bffff700 1 -- 172.21.15.22:0/2839476428 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c000eb80 con 0x7f30c80c9400 2023-07-06T01:20:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.030+0000 7f30bffff700 1 -- 172.21.15.22:0/2839476428 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0013420 con 0x7f30c80c9400 2023-07-06T01:20:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.030+0000 7f30cdde2700 1 -- 172.21.15.22:0/2839476428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c80c8c00 con 0x7f30c80c9400 2023-07-06T01:20:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.030+0000 7f30cdde2700 1 -- 172.21.15.22:0/2839476428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c80c7cc0 con 0x7f30c80c9400 2023-07-06T01:20:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.031+0000 7f30cdde2700 1 -- 172.21.15.22:0/2839476428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c80744b0 con 0x7f30c80c9400 2023-07-06T01:20:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.031+0000 7f30bffff700 1 -- 172.21.15.22:0/2839476428 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f30c0013580 con 0x7f30c80c9400 2023-07-06T01:20:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.032+0000 7f30bffff700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f30a806ace0 0x7f30a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.032+0000 7f30bffff700 1 -- 172.21.15.22:0/2839476428 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f30c00896b0 con 0x7f30c80c9400 2023-07-06T01:20:28.039 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.032+0000 7f30c77fe700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f30a806ace0 0x7f30a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.032+0000 7f30c77fe700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f30a806ace0 0x7f30a806d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f30b80090b0 tx=0x7f30b801f040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:28.040 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.036+0000 7f30bffff700 1 -- 172.21.15.22:0/2839476428 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f30c0054760 con 0x7f30c80c9400 2023-07-06T01:20:28.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.278+0000 7f30cdde2700 1 -- 172.21.15.22:0/2839476428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30c815f0b0 con 0x7f30c80c9400 2023-07-06T01:20:28.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.279+0000 7f30bffff700 1 -- 172.21.15.22:0/2839476428 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4671 (secure 0 0 0) 0x7f30c815f0b0 con 0x7f30c80c9400 2023-07-06T01:20:28.281 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:28.281 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/219695916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":219695916},{"type":"v1","addr":"172.21.15.103:6836","nonce":219695916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3102932229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":3102932229},{"type":"v1","addr":"172.21.15.103:6837","nonce":3102932229}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3787686782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3787686782},{"type":"v1","addr":"172.21.15.22:6834","nonce":3787686782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T01:20:24.257974+0000","modified":"2023-07-06T01:20:27.865213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6835/612827809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":612827809},{"type":"v1","addr":"172.21.15.22:6835","nonce":612827809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T01:20:28.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.281+0000 7f30cdde2700 1 -- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f30a806ace0 msgr2=0x7f30a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.281+0000 7f30cdde2700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f30a806ace0 0x7f30a806d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f30b80090b0 tx=0x7f30b801f040 comp rx=0 tx=0).stop 2023-07-06T01:20:28.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.281+0000 7f30cdde2700 1 -- 172.21.15.22:0/2839476428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30c80c9400 msgr2=0x7f30c80c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.281+0000 7f30cdde2700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30c80c9400 0x7f30c80c9880 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f30c80ca280 tx=0x7f30c000e440 comp rx=0 tx=0).stop 2023-07-06T01:20:28.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.281+0000 7f30cdde2700 1 -- 172.21.15.22:0/2839476428 shutdown_connections 2023-07-06T01:20:28.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.281+0000 7f30cdde2700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30c80c9400 0x7f30c80c9880 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.281+0000 7f30cdde2700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f30a806ace0 0x7f30a806d190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.281+0000 7f30cdde2700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f30b4003a30 0x7f30c81512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.281+0000 7f30cdde2700 1 --2- 172.21.15.22:0/2839476428 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f30c8151c00 0x7f30c80c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.281+0000 7f30cdde2700 1 -- 172.21.15.22:0/2839476428 >> 172.21.15.22:0/2839476428 conn(0x7f30c80b1890 msgr2=0x7f30c80b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:28.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.281+0000 7f30cdde2700 1 -- 172.21.15.22:0/2839476428 shutdown_connections 2023-07-06T01:20:28.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.281+0000 7f30cdde2700 1 -- 172.21.15.22:0/2839476428 wait complete. 2023-07-06T01:20:28.285 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 15 2023-07-06T01:20:28.295 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:20:28.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.417+0000 7f03c56fb700 1 Processor -- start 2023-07-06T01:20:28.419 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.417+0000 7f03c56fb700 1 -- start start 2023-07-06T01:20:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.418+0000 7f03c56fb700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f03c0159870 0x7f03c015df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.418+0000 7f03c56fb700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f03c00d0300 0x7f03c00ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03c56fb700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c00ce550 con 0x7f03c00715f0 2023-07-06T01:20:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03c56fb700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c00cdb60 con 0x7f03c0159870 2023-07-06T01:20:28.421 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03c56fb700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c015d490 con 0x7f03c00d0300 2023-07-06T01:20:28.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03beffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f03c0159870 0x7f03c015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03beffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f03c0159870 0x7f03c015df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47402/0 (socket says 172.21.15.22:47402) 2023-07-06T01:20:28.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03beffd700 1 -- 172.21.15.22:0/105767533 learned_addr learned my addr 172.21.15.22:0/105767533 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:28.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03bf7fe700 1 --2- 172.21.15.22:0/105767533 >> v2:172.21.15.103:3300/0 conn(0x7f03c00d0300 0x7f03c00ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03beffd700 1 -- 172.21.15.22:0/105767533 >> v2:172.21.15.103:3300/0 conn(0x7f03c00d0300 msgr2=0x7f03c00ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:28.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03beffd700 1 --2- 172.21.15.22:0/105767533 >> v2:172.21.15.103:3300/0 conn(0x7f03c00d0300 0x7f03c00ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03beffd700 1 -- 172.21.15.22:0/105767533 >> v1:172.21.15.103:6789/0 conn(0x7f03c00715f0 legacy=0x7f03c00cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:28.422 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03beffd700 1 -- 172.21.15.22:0/105767533 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c015d770 con 0x7f03c0159870 2023-07-06T01:20:28.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03bf7fe700 1 --2- 172.21.15.22:0/105767533 >> v2:172.21.15.103:3300/0 conn(0x7f03c00d0300 0x7f03c00ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:28.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03beffd700 1 --2- 172.21.15.22:0/105767533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c0159870 0x7f03c015df70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f03c00d06d0 tx=0x7f03b400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7752a51bc862d0bc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:28.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03bdffb700 1 -- 172.21.15.22:0/105767533 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b400dcf0 con 0x7f03c0159870 2023-07-06T01:20:28.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03bdffb700 1 -- 172.21.15.22:0/105767533 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b400de50 con 0x7f03c0159870 2023-07-06T01:20:28.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.419+0000 7f03bdffb700 1 -- 172.21.15.22:0/105767533 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4004250 con 0x7f03c0159870 2023-07-06T01:20:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.420+0000 7f03c56fb700 1 -- 172.21.15.22:0/105767533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c0159870 msgr2=0x7f03c015df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.420+0000 7f03c56fb700 1 --2- 172.21.15.22:0/105767533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c0159870 0x7f03c015df70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f03c00d06d0 tx=0x7f03b400be70 comp rx=0 tx=0).stop 2023-07-06T01:20:28.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.420+0000 7f03c56fb700 1 -- 172.21.15.22:0/105767533 shutdown_connections 2023-07-06T01:20:28.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.420+0000 7f03c56fb700 1 --2- 172.21.15.22:0/105767533 >> v2:172.21.15.103:3300/0 conn(0x7f03c00d0300 0x7f03c00ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.420+0000 7f03c56fb700 1 --2- 172.21.15.22:0/105767533 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c0159870 0x7f03c015df70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.420+0000 7f03c56fb700 1 -- 172.21.15.22:0/105767533 >> 172.21.15.22:0/105767533 conn(0x7f03c006b110 msgr2=0x7f03c006b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:28.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.420+0000 7f03c56fb700 1 -- 172.21.15.22:0/105767533 shutdown_connections 2023-07-06T01:20:28.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.420+0000 7f03c56fb700 1 -- 172.21.15.22:0/105767533 wait complete. 2023-07-06T01:20:28.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.421+0000 7f03c56fb700 1 Processor -- start 2023-07-06T01:20:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.421+0000 7f03c56fb700 1 -- start start 2023-07-06T01:20:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03c56fb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c00715f0 0x7f03c015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03c56fb700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c00d0300 0x7f03c014dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03c56fb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03c0159870 0x7f03c015afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03c56fb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c0159c40 con 0x7f03c00715f0 2023-07-06T01:20:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03c56fb700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c0159f20 con 0x7f03c00d0300 2023-07-06T01:20:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03c56fb700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c015a200 con 0x7f03c0159870 2023-07-06T01:20:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03beffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c00715f0 0x7f03c015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03beffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c00715f0 0x7f03c015c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47408/0 (socket says 172.21.15.22:47408) 2023-07-06T01:20:28.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03beffd700 1 -- 172.21.15.22:0/679908099 learned_addr learned my addr 172.21.15.22:0/679908099 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:28.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03be7fc700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c00d0300 0x7f03c014dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03bf7fe700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03c0159870 0x7f03c015afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03beffd700 1 -- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03c0159870 msgr2=0x7f03c015afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03beffd700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03c0159870 0x7f03c015afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03beffd700 1 -- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c00d0300 msgr2=0x7f03c014dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03beffd700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c00d0300 0x7f03c014dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03beffd700 1 -- 172.21.15.22:0/679908099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b400b7f0 con 0x7f03c00715f0 2023-07-06T01:20:28.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.422+0000 7f03bf7fe700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03c0159870 0x7f03c015afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:28.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.423+0000 7f03beffd700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c00715f0 0x7f03c015c6f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f03b4000c00 tx=0x7f03b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:28.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.423+0000 7f03a77fe700 1 -- 172.21.15.22:0/679908099 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4020070 con 0x7f03c00715f0 2023-07-06T01:20:28.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.423+0000 7f03a77fe700 1 -- 172.21.15.22:0/679908099 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b4004030 con 0x7f03c00715f0 2023-07-06T01:20:28.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.423+0000 7f03c56fb700 1 -- 172.21.15.22:0/679908099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c015a4e0 con 0x7f03c00715f0 2023-07-06T01:20:28.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.423+0000 7f03a77fe700 1 -- 172.21.15.22:0/679908099 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4010470 con 0x7f03c00715f0 2023-07-06T01:20:28.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.423+0000 7f03c56fb700 1 -- 172.21.15.22:0/679908099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c015aaa0 con 0x7f03c00715f0 2023-07-06T01:20:28.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.424+0000 7f03c56fb700 1 -- 172.21.15.22:0/679908099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c00744b0 con 0x7f03c00715f0 2023-07-06T01:20:28.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.424+0000 7f03a77fe700 1 -- 172.21.15.22:0/679908099 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f03b40193f0 con 0x7f03c00715f0 2023-07-06T01:20:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.425+0000 7f03a77fe700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f03ac06ace0 0x7f03ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.425+0000 7f03a77fe700 1 -- 172.21.15.22:0/679908099 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03b408c870 con 0x7f03c00715f0 2023-07-06T01:20:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.425+0000 7f03be7fc700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f03ac06ace0 0x7f03ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.425+0000 7f03be7fc700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f03ac06ace0 0x7f03ac06d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f03b0007fd0 tx=0x7f03b0007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.429+0000 7f03a77fe700 1 -- 172.21.15.22:0/679908099 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f03b4057890 con 0x7f03c00715f0 2023-07-06T01:20:28.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.626+0000 7f03c56fb700 1 -- 172.21.15.22:0/679908099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03c0073720 con 0x7f03c00715f0 2023-07-06T01:20:28.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.627+0000 7f03a77fe700 1 -- 172.21.15.22:0/679908099 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+18281 (secure 0 0 0) 0x7f03b4029020 con 0x7f03c00715f0 2023-07-06T01:20:28.629 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:28.630 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":34,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:20:25.831333+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:20:22.043370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T01:20:23.225298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:20:28.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.629+0000 7f03c56fb700 1 -- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f03ac06ace0 msgr2=0x7f03ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.629+0000 7f03c56fb700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f03ac06ace0 0x7f03ac06d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f03b0007fd0 tx=0x7f03b0007ec0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.629+0000 7f03c56fb700 1 -- 172.21.15.22:0/679908099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c00715f0 msgr2=0x7f03c015c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.629+0000 7f03c56fb700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c00715f0 0x7f03c015c6f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f03b4000c00 tx=0x7f03b4004750 comp rx=0 tx=0).stop 2023-07-06T01:20:28.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.629+0000 7f03c56fb700 1 -- 172.21.15.22:0/679908099 shutdown_connections 2023-07-06T01:20:28.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.629+0000 7f03c56fb700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03c0159870 0x7f03c015afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.629+0000 7f03c56fb700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03c00715f0 0x7f03c015c6f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.629+0000 7f03c56fb700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f03ac06ace0 0x7f03ac06d190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.629+0000 7f03c56fb700 1 --2- 172.21.15.22:0/679908099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03c00d0300 0x7f03c014dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.629+0000 7f03c56fb700 1 -- 172.21.15.22:0/679908099 >> 172.21.15.22:0/679908099 conn(0x7f03c006b110 msgr2=0x7f03c006a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:28.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.629+0000 7f03c56fb700 1 -- 172.21.15.22:0/679908099 shutdown_connections 2023-07-06T01:20:28.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.629+0000 7f03c56fb700 1 -- 172.21.15.22:0/679908099 wait complete. 2023-07-06T01:20:28.644 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T01:20:28.644 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:20:28.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.767+0000 7f8be4ebf700 1 Processor -- start 2023-07-06T01:20:28.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.767+0000 7f8be4ebf700 1 -- start start 2023-07-06T01:20:28.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.768+0000 7f8be4ebf700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8be0159360 0x7f8be015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.768+0000 7f8be4ebf700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8be00ccae0 0x7f8be0071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.768+0000 7f8be4ebf700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be00cf230 con 0x7f8be00cc660 2023-07-06T01:20:28.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.768+0000 7f8be4ebf700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be015df20 con 0x7f8be0159360 2023-07-06T01:20:28.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.768+0000 7f8be4ebf700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be00ce620 con 0x7f8be00ccae0 2023-07-06T01:20:28.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.768+0000 7f8bde59c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8be0159360 0x7f8be015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.768+0000 7f8bde59c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f8be0159360 0x7f8be015d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47416/0 (socket says 172.21.15.22:47416) 2023-07-06T01:20:28.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.768+0000 7f8bde59c700 1 -- 172.21.15.22:0/4142870148 learned_addr learned my addr 172.21.15.22:0/4142870148 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:28.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.769+0000 7f8bded9d700 1 --2- 172.21.15.22:0/4142870148 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8be00ccae0 0x7f8be0071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.769+0000 7f8bde59c700 1 -- 172.21.15.22:0/4142870148 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8be00ccae0 msgr2=0x7f8be0071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:28.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.769+0000 7f8bde59c700 1 --2- 172.21.15.22:0/4142870148 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8be00ccae0 0x7f8be0071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.769+0000 7f8bde59c700 1 -- 172.21.15.22:0/4142870148 >> v1:172.21.15.103:6789/0 conn(0x7f8be00cc660 legacy=0x7f8be0070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.769+0000 7f8bde59c700 1 -- 172.21.15.22:0/4142870148 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be00ce900 con 0x7f8be0159360 2023-07-06T01:20:28.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.769+0000 7f8bde59c700 1 --2- 172.21.15.22:0/4142870148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0159360 0x7f8be015d9e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40098f0 tx=0x7f8bd4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8aaff9c623ec9f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:28.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.769+0000 7f8bdd59a700 1 -- 172.21.15.22:0/4142870148 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400b8a0 con 0x7f8be0159360 2023-07-06T01:20:28.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.769+0000 7f8bdd59a700 1 -- 172.21.15.22:0/4142870148 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd400ba00 con 0x7f8be0159360 2023-07-06T01:20:28.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.769+0000 7f8bdd59a700 1 -- 172.21.15.22:0/4142870148 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400bd90 con 0x7f8be0159360 2023-07-06T01:20:28.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.770+0000 7f8be4ebf700 1 -- 172.21.15.22:0/4142870148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0159360 msgr2=0x7f8be015d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.770+0000 7f8be4ebf700 1 --2- 172.21.15.22:0/4142870148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0159360 0x7f8be015d9e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40098f0 tx=0x7f8bd4009c00 comp rx=0 tx=0).stop 2023-07-06T01:20:28.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.770+0000 7f8be4ebf700 1 -- 172.21.15.22:0/4142870148 shutdown_connections 2023-07-06T01:20:28.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.770+0000 7f8be4ebf700 1 --2- 172.21.15.22:0/4142870148 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8be00ccae0 0x7f8be0071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.770+0000 7f8be4ebf700 1 --2- 172.21.15.22:0/4142870148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0159360 0x7f8be015d9e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.770+0000 7f8be4ebf700 1 -- 172.21.15.22:0/4142870148 >> 172.21.15.22:0/4142870148 conn(0x7f8be00b1890 msgr2=0x7f8be00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:28.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.770+0000 7f8be4ebf700 1 -- 172.21.15.22:0/4142870148 shutdown_connections 2023-07-06T01:20:28.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.770+0000 7f8be4ebf700 1 -- 172.21.15.22:0/4142870148 wait complete. 2023-07-06T01:20:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.771+0000 7f8be4ebf700 1 Processor -- start 2023-07-06T01:20:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.771+0000 7f8be4ebf700 1 -- start start 2023-07-06T01:20:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8be4ebf700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8be00cc660 0x7f8be00b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8be4ebf700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8be00ccae0 0x7f8be006a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8be4ebf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0159360 0x7f8be00b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8be4ebf700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be00b30d0 con 0x7f8be0159360 2023-07-06T01:20:28.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8be4ebf700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be00b3370 con 0x7f8be00cc660 2023-07-06T01:20:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8be4ebf700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be00b3650 con 0x7f8be00ccae0 2023-07-06T01:20:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8bded9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0159360 0x7f8be00b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8bded9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0159360 0x7f8be00b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47426/0 (socket says 172.21.15.22:47426) 2023-07-06T01:20:28.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8bded9d700 1 -- 172.21.15.22:0/2143140337 learned_addr learned my addr 172.21.15.22:0/2143140337 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:28.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8bde59c700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8be00cc660 0x7f8be00b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8bddd9b700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8be00ccae0 0x7f8be006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8bded9d700 1 -- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8be00ccae0 msgr2=0x7f8be006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8bded9d700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8be00ccae0 0x7f8be006a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8bded9d700 1 -- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8be00cc660 msgr2=0x7f8be00b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8bded9d700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8be00cc660 0x7f8be00b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8bded9d700 1 -- 172.21.15.22:0/2143140337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd4009550 con 0x7f8be0159360 2023-07-06T01:20:28.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8bde59c700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8be00cc660 0x7f8be00b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:28.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.772+0000 7f8bded9d700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0159360 0x7f8be00b4410 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8007e40 tx=0x7f8bd800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:28.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.773+0000 7f8bceffd700 1 -- 172.21.15.22:0/2143140337 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd800eea0 con 0x7f8be0159360 2023-07-06T01:20:28.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.773+0000 7f8bceffd700 1 -- 172.21.15.22:0/2143140337 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd8014420 con 0x7f8be0159360 2023-07-06T01:20:28.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.773+0000 7f8be4ebf700 1 -- 172.21.15.22:0/2143140337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be00b3930 con 0x7f8be0159360 2023-07-06T01:20:28.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.773+0000 7f8bceffd700 1 -- 172.21.15.22:0/2143140337 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd80147b0 con 0x7f8be0159360 2023-07-06T01:20:28.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.773+0000 7f8be4ebf700 1 -- 172.21.15.22:0/2143140337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be00b3ef0 con 0x7f8be0159360 2023-07-06T01:20:28.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.774+0000 7f8be4ebf700 1 -- 172.21.15.22:0/2143140337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be00cf6a0 con 0x7f8be0159360 2023-07-06T01:20:28.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.774+0000 7f8bceffd700 1 -- 172.21.15.22:0/2143140337 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8bd8014910 con 0x7f8be0159360 2023-07-06T01:20:28.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.775+0000 7f8bceffd700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8bc006ace0 0x7f8bc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:28.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.775+0000 7f8bceffd700 1 -- 172.21.15.22:0/2143140337 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8bd8089d50 con 0x7f8be0159360 2023-07-06T01:20:28.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.775+0000 7f8bde59c700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8bc006ace0 0x7f8bc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:28.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.775+0000 7f8bde59c700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8bc006ace0 0x7f8bc006d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4014040 tx=0x7f8bd4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.779+0000 7f8bceffd700 1 -- 172.21.15.22:0/2143140337 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8be00cf6a0 con 0x7f8be0159360 2023-07-06T01:20:28.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.973+0000 7f8be4ebf700 1 -- 172.21.15.22:0/2143140337 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8be00ceec0 con 0x7f8bc006ace0 2023-07-06T01:20:28.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.983+0000 7f8bceffd700 1 -- 172.21.15.22:0/2143140337 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308073 (secure 0 0 0) 0x7f8bd401f070 con 0x7f8bc006ace0 2023-07-06T01:20:28.988 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:28.989 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.986+0000 7f8be4ebf700 1 -- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8bc006ace0 msgr2=0x7f8bc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.986+0000 7f8be4ebf700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8bc006ace0 0x7f8bc006d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4014040 tx=0x7f8bd4008030 comp rx=0 tx=0).stop 2023-07-06T01:20:28.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.986+0000 7f8be4ebf700 1 -- 172.21.15.22:0/2143140337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0159360 msgr2=0x7f8be00b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:28.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.986+0000 7f8be4ebf700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0159360 0x7f8be00b4410 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8007e40 tx=0x7f8bd800e3c0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.986+0000 7f8be4ebf700 1 -- 172.21.15.22:0/2143140337 shutdown_connections 2023-07-06T01:20:28.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.986+0000 7f8be4ebf700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8be0159360 0x7f8be00b4410 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.990 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.986+0000 7f8be4ebf700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f8bc006ace0 0x7f8bc006d190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.986+0000 7f8be4ebf700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8be00cc660 0x7f8be00b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.986+0000 7f8be4ebf700 1 --2- 172.21.15.22:0/2143140337 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8be00ccae0 0x7f8be006a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.986+0000 7f8be4ebf700 1 -- 172.21.15.22:0/2143140337 >> 172.21.15.22:0/2143140337 conn(0x7f8be00b1890 msgr2=0x7f8be006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.987+0000 7f8be4ebf700 1 -- 172.21.15.22:0/2143140337 shutdown_connections 2023-07-06T01:20:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:28.987+0000 7f8be4ebf700 1 -- 172.21.15.22:0/2143140337 wait complete. 2023-07-06T01:20:28.991 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:20:29.000 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-07-06T01:20:28.014418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":186,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219684,"kb_used_data":4068,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755036,"statfs":{"total":773094113280,"available":772869156864,"internally_reserved":0,"allocated":4165632,"data_stored":2386528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.189545"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855626+0000","last_change":"2023-07-06T01:20:24.852545+0000","last_active":"2023-07-06T01:20:24.855626+0000","last_peered":"2023-07-06T01:20:24.855626+0000","last_clean":"2023-07-06T01:20:24.855626+0000","last_became_active":"2023-07-06T01:20:24.852249+0000","last_became_peered":"2023-07-06T01:20:24.852249+0000","last_unstale":"2023-07-06T01:20:24.855626+0000","last_undegraded":"2023-07-06T01:20:24.855626+0000","last_fullsized":"2023-07-06T01:20:24.855626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:17.181181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.854187+0000","last_change":"2023-07-06T01:20:24.850725+0000","last_active":"2023-07-06T01:20:24.854187+0000","last_peered":"2023-07-06T01:20:24.854187+0000","last_clean":"2023-07-06T01:20:24.854187+0000","last_became_active":"2023-07-06T01:20:24.849359+0000","last_became_peered":"2023-07-06T01:20:24.849359+0000","last_unstale":"2023-07-06T01:20:24.854187+0000","last_undegraded":"2023-07-06T01:20:24.854187+0000","last_fullsized":"2023-07-06T01:20:24.854187+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:20.409326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834789+0000","last_change":"2023-07-06T01:20:24.855847+0000","last_active":"2023-07-06T01:20:25.834789+0000","last_peered":"2023-07-06T01:20:25.834789+0000","last_clean":"2023-07-06T01:20:25.834789+0000","last_became_active":"2023-07-06T01:20:24.855571+0000","last_became_peered":"2023-07-06T01:20:24.855571+0000","last_unstale":"2023-07-06T01:20:25.834789+0000","last_undegraded":"2023-07-06T01:20:25.834789+0000","last_fullsized":"2023-07-06T01:20:25.834789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884281+0000","last_change":"2023-07-06T01:20:24.860708+0000","last_active":"2023-07-06T01:20:25.884281+0000","last_peered":"2023-07-06T01:20:25.884281+0000","last_clean":"2023-07-06T01:20:25.884281+0000","last_became_active":"2023-07-06T01:20:24.860220+0000","last_became_peered":"2023-07-06T01:20:24.860220+0000","last_unstale":"2023-07-06T01:20:25.884281+0000","last_undegraded":"2023-07-06T01:20:25.884281+0000","last_fullsized":"2023-07-06T01:20:25.884281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:42:33.239141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884127+0000","last_change":"2023-07-06T01:20:24.850342+0000","last_active":"2023-07-06T01:20:25.884127+0000","last_peered":"2023-07-06T01:20:25.884127+0000","last_clean":"2023-07-06T01:20:25.884127+0000","last_became_active":"2023-07-06T01:20:24.850083+0000","last_became_peered":"2023-07-06T01:20:24.850083+0000","last_unstale":"2023-07-06T01:20:25.884127+0000","last_undegraded":"2023-07-06T01:20:25.884127+0000","last_fullsized":"2023-07-06T01:20:25.884127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:46:35.434349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.850816+0000","last_change":"2023-07-06T01:20:24.848000+0000","last_active":"2023-07-06T01:20:24.850816+0000","last_peered":"2023-07-06T01:20:24.850816+0000","last_clean":"2023-07-06T01:20:24.850816+0000","last_became_active":"2023-07-06T01:20:24.847814+0000","last_became_peered":"2023-07-06T01:20:24.847814+0000","last_unstale":"2023-07-06T01:20:24.850816+0000","last_undegraded":"2023-07-06T01:20:24.850816+0000","last_fullsized":"2023-07-06T01:20:24.850816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:51.766923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886546+0000","last_change":"2023-07-06T01:20:24.856730+0000","last_active":"2023-07-06T01:20:25.886546+0000","last_peered":"2023-07-06T01:20:25.886546+0000","last_clean":"2023-07-06T01:20:25.886546+0000","last_became_active":"2023-07-06T01:20:24.856460+0000","last_became_peered":"2023-07-06T01:20:24.856460+0000","last_unstale":"2023-07-06T01:20:25.886546+0000","last_undegraded":"2023-07-06T01:20:25.886546+0000","last_fullsized":"2023-07-06T01:20:25.886546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:52.466998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859689+0000","last_change":"2023-07-06T01:20:24.858297+0000","last_active":"2023-07-06T01:20:24.859689+0000","last_peered":"2023-07-06T01:20:24.859689+0000","last_clean":"2023-07-06T01:20:24.859689+0000","last_became_active":"2023-07-06T01:20:24.858033+0000","last_became_peered":"2023-07-06T01:20:24.858033+0000","last_unstale":"2023-07-06T01:20:24.859689+0000","last_undegraded":"2023-07-06T01:20:24.859689+0000","last_fullsized":"2023-07-06T01:20:24.859689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:38.837442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834859+0000","last_change":"2023-07-06T01:20:24.859281+0000","last_active":"2023-07-06T01:20:25.834859+0000","last_peered":"2023-07-06T01:20:25.834859+0000","last_clean":"2023-07-06T01:20:25.834859+0000","last_became_active":"2023-07-06T01:20:24.858982+0000","last_became_peered":"2023-07-06T01:20:24.858982+0000","last_unstale":"2023-07-06T01:20:25.834859+0000","last_undegraded":"2023-07-06T01:20:25.834859+0000","last_fullsized":"2023-07-06T01:20:25.834859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:41.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886707+0000","last_change":"2023-07-06T01:20:24.854480+0000","last_active":"2023-07-06T01:20:25.886707+0000","last_peered":"2023-07-06T01:20:25.886707+0000","last_clean":"2023-07-06T01:20:25.886707+0000","last_became_active":"2023-07-06T01:20:24.854116+0000","last_became_peered":"2023-07-06T01:20:24.854116+0000","last_unstale":"2023-07-06T01:20:25.886707+0000","last_undegraded":"2023-07-06T01:20:25.886707+0000","last_fullsized":"2023-07-06T01:20:25.886707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:12.103457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886974+0000","last_change":"2023-07-06T01:20:24.854685+0000","last_active":"2023-07-06T01:20:25.886974+0000","last_peered":"2023-07-06T01:20:25.886974+0000","last_clean":"2023-07-06T01:20:25.886974+0000","last_became_active":"2023-07-06T01:20:24.853605+0000","last_became_peered":"2023-07-06T01:20:24.853605+0000","last_unstale":"2023-07-06T01:20:25.886974+0000","last_undegraded":"2023-07-06T01:20:25.886974+0000","last_fullsized":"2023-07-06T01:20:25.886974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:54:12.127760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834625+0000","last_change":"2023-07-06T01:20:24.853728+0000","last_active":"2023-07-06T01:20:25.834625+0000","last_peered":"2023-07-06T01:20:25.834625+0000","last_clean":"2023-07-06T01:20:25.834625+0000","last_became_active":"2023-07-06T01:20:24.853427+0000","last_became_peered":"2023-07-06T01:20:24.853427+0000","last_unstale":"2023-07-06T01:20:25.834625+0000","last_undegraded":"2023-07-06T01:20:25.834625+0000","last_fullsized":"2023-07-06T01:20:25.834625+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:31.557538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886831+0000","last_change":"2023-07-06T01:20:24.856012+0000","last_active":"2023-07-06T01:20:25.886831+0000","last_peered":"2023-07-06T01:20:25.886831+0000","last_clean":"2023-07-06T01:20:25.886831+0000","last_became_active":"2023-07-06T01:20:24.855762+0000","last_became_peered":"2023-07-06T01:20:24.855762+0000","last_unstale":"2023-07-06T01:20:25.886831+0000","last_undegraded":"2023-07-06T01:20:25.886831+0000","last_fullsized":"2023-07-06T01:20:25.886831+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:45.220808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886397+0000","last_change":"2023-07-06T01:20:24.857423+0000","last_active":"2023-07-06T01:20:25.886397+0000","last_peered":"2023-07-06T01:20:25.886397+0000","last_clean":"2023-07-06T01:20:25.886397+0000","last_became_active":"2023-07-06T01:20:24.857164+0000","last_became_peered":"2023-07-06T01:20:24.857164+0000","last_unstale":"2023-07-06T01:20:25.886397+0000","last_undegraded":"2023-07-06T01:20:25.886397+0000","last_fullsized":"2023-07-06T01:20:25.886397+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836700+0000","last_change":"2023-07-06T01:20:24.860237+0000","last_active":"2023-07-06T01:20:25.836700+0000","last_peered":"2023-07-06T01:20:25.836700+0000","last_clean":"2023-07-06T01:20:25.836700+0000","last_became_active":"2023-07-06T01:20:24.859324+0000","last_became_peered":"2023-07-06T01:20:24.859324+0000","last_unstale":"2023-07-06T01:20:25.836700+0000","last_undegraded":"2023-07-06T01:20:25.836700+0000","last_fullsized":"2023-07-06T01:20:25.836700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:50.731526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884196+0000","last_change":"2023-07-06T01:20:24.852638+0000","last_active":"2023-07-06T01:20:25.884196+0000","last_peered":"2023-07-06T01:20:25.884196+0000","last_clean":"2023-07-06T01:20:25.884196+0000","last_became_active":"2023-07-06T01:20:24.852341+0000","last_became_peered":"2023-07-06T01:20:24.852341+0000","last_unstale":"2023-07-06T01:20:25.884196+0000","last_undegraded":"2023-07-06T01:20:25.884196+0000","last_fullsized":"2023-07-06T01:20:25.884196+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:17:34.632929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836872+0000","last_change":"2023-07-06T01:20:24.859626+0000","last_active":"2023-07-06T01:20:25.836872+0000","last_peered":"2023-07-06T01:20:25.836872+0000","last_clean":"2023-07-06T01:20:25.836872+0000","last_became_active":"2023-07-06T01:20:24.859274+0000","last_became_peered":"2023-07-06T01:20:24.859274+0000","last_unstale":"2023-07-06T01:20:25.836872+0000","last_undegraded":"2023-07-06T01:20:25.836872+0000","last_fullsized":"2023-07-06T01:20:25.836872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:03:24.140950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886696+0000","last_change":"2023-07-06T01:20:24.854443+0000","last_active":"2023-07-06T01:20:25.886696+0000","last_peered":"2023-07-06T01:20:25.886696+0000","last_clean":"2023-07-06T01:20:25.886696+0000","last_became_active":"2023-07-06T01:20:24.853256+0000","last_became_peered":"2023-07-06T01:20:24.853256+0000","last_unstale":"2023-07-06T01:20:25.886696+0000","last_undegraded":"2023-07-06T01:20:25.886696+0000","last_fullsized":"2023-07-06T01:20:25.886696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:20.409326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884398+0000","last_change":"2023-07-06T01:20:24.852693+0000","last_active":"2023-07-06T01:20:25.884398+0000","last_peered":"2023-07-06T01:20:25.884398+0000","last_clean":"2023-07-06T01:20:25.884398+0000","last_became_active":"2023-07-06T01:20:24.852395+0000","last_became_peered":"2023-07-06T01:20:24.852395+0000","last_unstale":"2023-07-06T01:20:25.884398+0000","last_undegraded":"2023-07-06T01:20:25.884398+0000","last_fullsized":"2023-07-06T01:20:25.884398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:35.585237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836707+0000","last_change":"2023-07-06T01:20:24.856089+0000","last_active":"2023-07-06T01:20:25.836707+0000","last_peered":"2023-07-06T01:20:25.836707+0000","last_clean":"2023-07-06T01:20:25.836707+0000","last_became_active":"2023-07-06T01:20:24.855834+0000","last_became_peered":"2023-07-06T01:20:24.855834+0000","last_unstale":"2023-07-06T01:20:25.836707+0000","last_undegraded":"2023-07-06T01:20:25.836707+0000","last_fullsized":"2023-07-06T01:20:25.836707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:03.093141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859016+0000","last_change":"2023-07-06T01:20:24.856754+0000","last_active":"2023-07-06T01:20:24.859016+0000","last_peered":"2023-07-06T01:20:24.859016+0000","last_clean":"2023-07-06T01:20:24.859016+0000","last_became_active":"2023-07-06T01:20:24.856396+0000","last_became_peered":"2023-07-06T01:20:24.856396+0000","last_unstale":"2023-07-06T01:20:24.859016+0000","last_undegraded":"2023-07-06T01:20:24.859016+0000","last_fullsized":"2023-07-06T01:20:24.859016+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:06:02.668449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835952+0000","last_change":"2023-07-06T01:20:24.854996+0000","last_active":"2023-07-06T01:20:25.835952+0000","last_peered":"2023-07-06T01:20:25.835952+0000","last_clean":"2023-07-06T01:20:25.835952+0000","last_became_active":"2023-07-06T01:20:24.854737+0000","last_became_peered":"2023-07-06T01:20:24.854737+0000","last_unstale":"2023-07-06T01:20:25.835952+0000","last_undegraded":"2023-07-06T01:20:25.835952+0000","last_fullsized":"2023-07-06T01:20:25.835952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:51.766923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886641+0000","last_change":"2023-07-06T01:20:24.856804+0000","last_active":"2023-07-06T01:20:25.886641+0000","last_peered":"2023-07-06T01:20:25.886641+0000","last_clean":"2023-07-06T01:20:25.886641+0000","last_became_active":"2023-07-06T01:20:24.856490+0000","last_became_peered":"2023-07-06T01:20:24.856490+0000","last_unstale":"2023-07-06T01:20:25.886641+0000","last_undegraded":"2023-07-06T01:20:25.886641+0000","last_fullsized":"2023-07-06T01:20:25.886641+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:23:07.815642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835630+0000","last_change":"2023-07-06T01:20:24.850799+0000","last_active":"2023-07-06T01:20:25.835630+0000","last_peered":"2023-07-06T01:20:25.835630+0000","last_clean":"2023-07-06T01:20:25.835630+0000","last_became_active":"2023-07-06T01:20:24.850463+0000","last_became_peered":"2023-07-06T01:20:24.850463+0000","last_unstale":"2023-07-06T01:20:25.835630+0000","last_undegraded":"2023-07-06T01:20:25.835630+0000","last_fullsized":"2023-07-06T01:20:25.835630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:50.421447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835944+0000","last_change":"2023-07-06T01:20:24.857969+0000","last_active":"2023-07-06T01:20:25.835944+0000","last_peered":"2023-07-06T01:20:25.835944+0000","last_clean":"2023-07-06T01:20:25.835944+0000","last_became_active":"2023-07-06T01:20:24.857674+0000","last_became_peered":"2023-07-06T01:20:24.857674+0000","last_unstale":"2023-07-06T01:20:25.835944+0000","last_undegraded":"2023-07-06T01:20:25.835944+0000","last_fullsized":"2023-07-06T01:20:25.835944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:57.090630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.856018+0000","last_change":"2023-07-06T01:20:24.854034+0000","last_active":"2023-07-06T01:20:24.856018+0000","last_peered":"2023-07-06T01:20:24.856018+0000","last_clean":"2023-07-06T01:20:24.856018+0000","last_became_active":"2023-07-06T01:20:24.853820+0000","last_became_peered":"2023-07-06T01:20:24.853820+0000","last_unstale":"2023-07-06T01:20:24.856018+0000","last_undegraded":"2023-07-06T01:20:24.856018+0000","last_fullsized":"2023-07-06T01:20:24.856018+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:57.090630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836884+0000","last_change":"2023-07-06T01:20:23.849203+0000","last_active":"2023-07-06T01:20:25.836884+0000","last_peered":"2023-07-06T01:20:25.836884+0000","last_clean":"2023-07-06T01:20:25.836884+0000","last_became_active":"2023-07-06T01:20:23.848883+0000","last_became_peered":"2023-07-06T01:20:23.848883+0000","last_unstale":"2023-07-06T01:20:25.836884+0000","last_undegraded":"2023-07-06T01:20:25.836884+0000","last_fullsized":"2023-07-06T01:20:25.836884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:06.770007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.860470+0000","last_change":"2023-07-06T01:20:24.859359+0000","last_active":"2023-07-06T01:20:24.860470+0000","last_peered":"2023-07-06T01:20:24.860470+0000","last_clean":"2023-07-06T01:20:24.860470+0000","last_became_active":"2023-07-06T01:20:24.859152+0000","last_became_peered":"2023-07-06T01:20:24.859152+0000","last_unstale":"2023-07-06T01:20:24.860470+0000","last_undegraded":"2023-07-06T01:20:24.860470+0000","last_fullsized":"2023-07-06T01:20:24.860470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:01.181469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834689+0000","last_change":"2023-07-06T01:20:23.845980+0000","last_active":"2023-07-06T01:20:25.834689+0000","last_peered":"2023-07-06T01:20:25.834689+0000","last_clean":"2023-07-06T01:20:25.834689+0000","last_became_active":"2023-07-06T01:20:23.845696+0000","last_became_peered":"2023-07-06T01:20:23.845696+0000","last_unstale":"2023-07-06T01:20:25.834689+0000","last_undegraded":"2023-07-06T01:20:25.834689+0000","last_fullsized":"2023-07-06T01:20:25.834689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:58.579906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.858947+0000","last_change":"2023-07-06T01:20:23.846742+0000","last_active":"2023-07-06T01:20:24.858947+0000","last_peered":"2023-07-06T01:20:24.858947+0000","last_clean":"2023-07-06T01:20:24.858947+0000","last_became_active":"2023-07-06T01:20:23.846440+0000","last_became_peered":"2023-07-06T01:20:23.846440+0000","last_unstale":"2023-07-06T01:20:24.858947+0000","last_undegraded":"2023-07-06T01:20:24.858947+0000","last_fullsized":"2023-07-06T01:20:24.858947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:06.770007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862194+0000","last_change":"2023-07-06T01:20:24.861558+0000","last_active":"2023-07-06T01:20:24.862194+0000","last_peered":"2023-07-06T01:20:24.862194+0000","last_clean":"2023-07-06T01:20:24.862194+0000","last_became_active":"2023-07-06T01:20:24.860857+0000","last_became_peered":"2023-07-06T01:20:24.860857+0000","last_unstale":"2023-07-06T01:20:24.862194+0000","last_undegraded":"2023-07-06T01:20:24.862194+0000","last_fullsized":"2023-07-06T01:20:24.862194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:52.466998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.827224+0000","last_change":"2023-07-06T01:20:22.827224+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.827224+0000","last_undegraded":"2023-07-06T01:20:22.827224+0000","last_fullsized":"2023-07-06T01:20:22.827224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834373+0000","last_change":"2023-07-06T01:20:24.839527+0000","last_active":"2023-07-06T01:20:25.834373+0000","last_peered":"2023-07-06T01:20:25.834373+0000","last_clean":"2023-07-06T01:20:25.834373+0000","last_became_active":"2023-07-06T01:20:24.839201+0000","last_became_peered":"2023-07-06T01:20:24.839201+0000","last_unstale":"2023-07-06T01:20:25.834373+0000","last_undegraded":"2023-07-06T01:20:25.834373+0000","last_fullsized":"2023-07-06T01:20:25.834373+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:57:20.290658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886526+0000","last_change":"2023-07-06T01:20:23.853509+0000","last_active":"2023-07-06T01:20:25.886526+0000","last_peered":"2023-07-06T01:20:25.886526+0000","last_clean":"2023-07-06T01:20:25.886526+0000","last_became_active":"2023-07-06T01:20:23.853060+0000","last_became_peered":"2023-07-06T01:20:23.853060+0000","last_unstale":"2023-07-06T01:20:25.886526+0000","last_undegraded":"2023-07-06T01:20:25.886526+0000","last_fullsized":"2023-07-06T01:20:25.886526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:40.214108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886380+0000","last_change":"2023-07-06T01:20:24.856240+0000","last_active":"2023-07-06T01:20:25.886380+0000","last_peered":"2023-07-06T01:20:25.886380+0000","last_clean":"2023-07-06T01:20:25.886380+0000","last_became_active":"2023-07-06T01:20:24.853426+0000","last_became_peered":"2023-07-06T01:20:24.853426+0000","last_unstale":"2023-07-06T01:20:25.886380+0000","last_undegraded":"2023-07-06T01:20:25.886380+0000","last_fullsized":"2023-07-06T01:20:25.886380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:59.584568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.829932+0000","last_change":"2023-07-06T01:20:22.829932+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.829932+0000","last_undegraded":"2023-07-06T01:20:22.829932+0000","last_fullsized":"2023-07-06T01:20:22.829932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836572+0000","last_change":"2023-07-06T01:20:23.849040+0000","last_active":"2023-07-06T01:20:24.836572+0000","last_peered":"2023-07-06T01:20:24.836572+0000","last_clean":"2023-07-06T01:20:24.836572+0000","last_became_active":"2023-07-06T01:20:23.848756+0000","last_became_peered":"2023-07-06T01:20:23.848756+0000","last_unstale":"2023-07-06T01:20:24.836572+0000","last_undegraded":"2023-07-06T01:20:24.836572+0000","last_fullsized":"2023-07-06T01:20:24.836572+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:05:00.225120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855846+0000","last_change":"2023-07-06T01:20:24.849751+0000","last_active":"2023-07-06T01:20:24.855846+0000","last_peered":"2023-07-06T01:20:24.855846+0000","last_clean":"2023-07-06T01:20:24.855846+0000","last_became_active":"2023-07-06T01:20:24.849475+0000","last_became_peered":"2023-07-06T01:20:24.849475+0000","last_unstale":"2023-07-06T01:20:24.855846+0000","last_undegraded":"2023-07-06T01:20:24.855846+0000","last_fullsized":"2023-07-06T01:20:24.855846+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.826762+0000","last_change":"2023-07-06T01:20:22.826762+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.826762+0000","last_undegraded":"2023-07-06T01:20:22.826762+0000","last_fullsized":"2023-07-06T01:20:22.826762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884424+0000","last_change":"2023-07-06T01:20:24.860540+0000","last_active":"2023-07-06T01:20:25.884424+0000","last_peered":"2023-07-06T01:20:25.884424+0000","last_clean":"2023-07-06T01:20:25.884424+0000","last_became_active":"2023-07-06T01:20:24.859869+0000","last_became_peered":"2023-07-06T01:20:24.859869+0000","last_unstale":"2023-07-06T01:20:25.884424+0000","last_undegraded":"2023-07-06T01:20:25.884424+0000","last_fullsized":"2023-07-06T01:20:25.884424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:23.281099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.845141+0000","last_change":"2023-07-06T01:20:23.848957+0000","last_active":"2023-07-06T01:20:24.845141+0000","last_peered":"2023-07-06T01:20:24.845141+0000","last_clean":"2023-07-06T01:20:24.845141+0000","last_became_active":"2023-07-06T01:20:23.848646+0000","last_became_peered":"2023-07-06T01:20:23.848646+0000","last_unstale":"2023-07-06T01:20:24.845141+0000","last_undegraded":"2023-07-06T01:20:24.845141+0000","last_fullsized":"2023-07-06T01:20:24.845141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:35:50.683696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886834+0000","last_change":"2023-07-06T01:20:23.847858+0000","last_active":"2023-07-06T01:20:25.886834+0000","last_peered":"2023-07-06T01:20:25.886834+0000","last_clean":"2023-07-06T01:20:25.886834+0000","last_became_active":"2023-07-06T01:20:23.847549+0000","last_became_peered":"2023-07-06T01:20:23.847549+0000","last_unstale":"2023-07-06T01:20:25.886834+0000","last_undegraded":"2023-07-06T01:20:25.886834+0000","last_fullsized":"2023-07-06T01:20:25.886834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:53:56.761538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834565+0000","last_change":"2023-07-06T01:20:24.848275+0000","last_active":"2023-07-06T01:20:25.834565+0000","last_peered":"2023-07-06T01:20:25.834565+0000","last_clean":"2023-07-06T01:20:25.834565+0000","last_became_active":"2023-07-06T01:20:24.847949+0000","last_became_peered":"2023-07-06T01:20:24.847949+0000","last_unstale":"2023-07-06T01:20:25.834565+0000","last_undegraded":"2023-07-06T01:20:25.834565+0000","last_fullsized":"2023-07-06T01:20:25.834565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:24:03.775615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887068+0000","last_change":"2023-07-06T01:20:23.849878+0000","last_active":"2023-07-06T01:20:25.887068+0000","last_peered":"2023-07-06T01:20:25.887068+0000","last_clean":"2023-07-06T01:20:25.887068+0000","last_became_active":"2023-07-06T01:20:23.849573+0000","last_became_peered":"2023-07-06T01:20:23.849573+0000","last_unstale":"2023-07-06T01:20:25.887068+0000","last_undegraded":"2023-07-06T01:20:25.887068+0000","last_fullsized":"2023-07-06T01:20:25.887068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:12:26.209870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.868009+0000","last_change":"2023-07-06T01:20:23.845648+0000","last_active":"2023-07-06T01:20:24.868009+0000","last_peered":"2023-07-06T01:20:24.868009+0000","last_clean":"2023-07-06T01:20:24.868009+0000","last_became_active":"2023-07-06T01:20:23.845340+0000","last_became_peered":"2023-07-06T01:20:23.845340+0000","last_unstale":"2023-07-06T01:20:24.868009+0000","last_undegraded":"2023-07-06T01:20:24.868009+0000","last_fullsized":"2023-07-06T01:20:24.868009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:39:35.605373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834521+0000","last_change":"2023-07-06T01:20:24.858282+0000","last_active":"2023-07-06T01:20:25.834521+0000","last_peered":"2023-07-06T01:20:25.834521+0000","last_clean":"2023-07-06T01:20:25.834521+0000","last_became_active":"2023-07-06T01:20:24.857879+0000","last_became_peered":"2023-07-06T01:20:24.857879+0000","last_unstale":"2023-07-06T01:20:25.834521+0000","last_undegraded":"2023-07-06T01:20:25.834521+0000","last_fullsized":"2023-07-06T01:20:25.834521+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:22:46.902965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855788+0000","last_change":"2023-07-06T01:20:23.841745+0000","last_active":"2023-07-06T01:20:24.855788+0000","last_peered":"2023-07-06T01:20:24.855788+0000","last_clean":"2023-07-06T01:20:24.855788+0000","last_became_active":"2023-07-06T01:20:23.841438+0000","last_became_peered":"2023-07-06T01:20:23.841438+0000","last_unstale":"2023-07-06T01:20:24.855788+0000","last_undegraded":"2023-07-06T01:20:24.855788+0000","last_fullsized":"2023-07-06T01:20:24.855788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:54:11.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855590+0000","last_change":"2023-07-06T01:20:24.854272+0000","last_active":"2023-07-06T01:20:24.855590+0000","last_peered":"2023-07-06T01:20:24.855590+0000","last_clean":"2023-07-06T01:20:24.855590+0000","last_became_active":"2023-07-06T01:20:24.854002+0000","last_became_peered":"2023-07-06T01:20:24.854002+0000","last_unstale":"2023-07-06T01:20:24.855590+0000","last_undegraded":"2023-07-06T01:20:24.855590+0000","last_fullsized":"2023-07-06T01:20:24.855590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:23:07.815642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836795+0000","last_change":"2023-07-06T01:20:23.854212+0000","last_active":"2023-07-06T01:20:24.836795+0000","last_peered":"2023-07-06T01:20:24.836795+0000","last_clean":"2023-07-06T01:20:24.836795+0000","last_became_active":"2023-07-06T01:20:23.853928+0000","last_became_peered":"2023-07-06T01:20:23.853928+0000","last_unstale":"2023-07-06T01:20:24.836795+0000","last_undegraded":"2023-07-06T01:20:24.836795+0000","last_fullsized":"2023-07-06T01:20:24.836795+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:49.726864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835969+0000","last_change":"2023-07-06T01:20:23.847446+0000","last_active":"2023-07-06T01:20:25.835969+0000","last_peered":"2023-07-06T01:20:25.835969+0000","last_clean":"2023-07-06T01:20:25.835969+0000","last_became_active":"2023-07-06T01:20:23.846929+0000","last_became_peered":"2023-07-06T01:20:23.846929+0000","last_unstale":"2023-07-06T01:20:25.835969+0000","last_undegraded":"2023-07-06T01:20:25.835969+0000","last_fullsized":"2023-07-06T01:20:25.835969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:05:00.225120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855866+0000","last_change":"2023-07-06T01:20:24.855191+0000","last_active":"2023-07-06T01:20:24.855866+0000","last_peered":"2023-07-06T01:20:24.855866+0000","last_clean":"2023-07-06T01:20:24.855866+0000","last_became_active":"2023-07-06T01:20:24.854898+0000","last_became_peered":"2023-07-06T01:20:24.854898+0000","last_unstale":"2023-07-06T01:20:24.855866+0000","last_undegraded":"2023-07-06T01:20:24.855866+0000","last_fullsized":"2023-07-06T01:20:24.855866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.419832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.829961+0000","last_change":"2023-07-06T01:20:22.829961+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.829961+0000","last_undegraded":"2023-07-06T01:20:22.829961+0000","last_fullsized":"2023-07-06T01:20:22.829961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862166+0000","last_change":"2023-07-06T01:20:24.861072+0000","last_active":"2023-07-06T01:20:24.862166+0000","last_peered":"2023-07-06T01:20:24.862166+0000","last_clean":"2023-07-06T01:20:24.862166+0000","last_became_active":"2023-07-06T01:20:24.860745+0000","last_became_peered":"2023-07-06T01:20:24.860745+0000","last_unstale":"2023-07-06T01:20:24.862166+0000","last_undegraded":"2023-07-06T01:20:24.862166+0000","last_fullsized":"2023-07-06T01:20:24.862166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:45.220808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887003+0000","last_change":"2023-07-06T01:20:23.848801+0000","last_active":"2023-07-06T01:20:25.887003+0000","last_peered":"2023-07-06T01:20:25.887003+0000","last_clean":"2023-07-06T01:20:25.887003+0000","last_became_active":"2023-07-06T01:20:23.848412+0000","last_became_peered":"2023-07-06T01:20:23.848412+0000","last_unstale":"2023-07-06T01:20:25.887003+0000","last_undegraded":"2023-07-06T01:20:25.887003+0000","last_fullsized":"2023-07-06T01:20:25.887003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:35:50.683696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886952+0000","last_change":"2023-07-06T01:20:24.854853+0000","last_active":"2023-07-06T01:20:25.886952+0000","last_peered":"2023-07-06T01:20:25.886952+0000","last_clean":"2023-07-06T01:20:25.886952+0000","last_became_active":"2023-07-06T01:20:24.854159+0000","last_became_peered":"2023-07-06T01:20:24.854159+0000","last_unstale":"2023-07-06T01:20:25.886952+0000","last_undegraded":"2023-07-06T01:20:25.886952+0000","last_fullsized":"2023-07-06T01:20:25.886952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.419832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.826760+0000","last_change":"2023-07-06T01:20:22.826760+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.826760+0000","last_undegraded":"2023-07-06T01:20:22.826760+0000","last_fullsized":"2023-07-06T01:20:22.826760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884278+0000","last_change":"2023-07-06T01:20:23.846883+0000","last_active":"2023-07-06T01:20:25.884278+0000","last_peered":"2023-07-06T01:20:25.884278+0000","last_clean":"2023-07-06T01:20:25.884278+0000","last_became_active":"2023-07-06T01:20:23.846566+0000","last_became_peered":"2023-07-06T01:20:23.846566+0000","last_unstale":"2023-07-06T01:20:25.884278+0000","last_undegraded":"2023-07-06T01:20:25.884278+0000","last_fullsized":"2023-07-06T01:20:25.884278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:44:49.870273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884313+0000","last_change":"2023-07-06T01:20:24.853783+0000","last_active":"2023-07-06T01:20:25.884313+0000","last_peered":"2023-07-06T01:20:25.884313+0000","last_clean":"2023-07-06T01:20:25.884313+0000","last_became_active":"2023-07-06T01:20:24.853427+0000","last_became_peered":"2023-07-06T01:20:24.853427+0000","last_unstale":"2023-07-06T01:20:25.884313+0000","last_undegraded":"2023-07-06T01:20:25.884313+0000","last_fullsized":"2023-07-06T01:20:25.884313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:33:25.311596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.837321+0000","last_change":"2023-07-06T01:20:23.847016+0000","last_active":"2023-07-06T01:20:24.837321+0000","last_peered":"2023-07-06T01:20:24.837321+0000","last_clean":"2023-07-06T01:20:24.837321+0000","last_became_active":"2023-07-06T01:20:23.846748+0000","last_became_peered":"2023-07-06T01:20:23.846748+0000","last_unstale":"2023-07-06T01:20:24.837321+0000","last_undegraded":"2023-07-06T01:20:24.837321+0000","last_fullsized":"2023-07-06T01:20:24.837321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:13.424430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884268+0000","last_change":"2023-07-06T01:20:23.847882+0000","last_active":"2023-07-06T01:20:25.884268+0000","last_peered":"2023-07-06T01:20:25.884268+0000","last_clean":"2023-07-06T01:20:25.884268+0000","last_became_active":"2023-07-06T01:20:23.847534+0000","last_became_peered":"2023-07-06T01:20:23.847534+0000","last_unstale":"2023-07-06T01:20:25.884268+0000","last_undegraded":"2023-07-06T01:20:25.884268+0000","last_fullsized":"2023-07-06T01:20:25.884268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:22:12.321678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834687+0000","last_change":"2023-07-06T01:20:24.848033+0000","last_active":"2023-07-06T01:20:25.834687+0000","last_peered":"2023-07-06T01:20:25.834687+0000","last_clean":"2023-07-06T01:20:25.834687+0000","last_became_active":"2023-07-06T01:20:24.847514+0000","last_became_peered":"2023-07-06T01:20:24.847514+0000","last_unstale":"2023-07-06T01:20:25.834687+0000","last_undegraded":"2023-07-06T01:20:25.834687+0000","last_fullsized":"2023-07-06T01:20:25.834687+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:02.805903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884363+0000","last_change":"2023-07-06T01:20:23.850960+0000","last_active":"2023-07-06T01:20:25.884363+0000","last_peered":"2023-07-06T01:20:25.884363+0000","last_clean":"2023-07-06T01:20:25.884363+0000","last_became_active":"2023-07-06T01:20:23.850699+0000","last_became_peered":"2023-07-06T01:20:23.850699+0000","last_unstale":"2023-07-06T01:20:25.884363+0000","last_undegraded":"2023-07-06T01:20:25.884363+0000","last_fullsized":"2023-07-06T01:20:25.884363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:30:39.863076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886519+0000","last_change":"2023-07-06T01:20:24.861071+0000","last_active":"2023-07-06T01:20:25.886519+0000","last_peered":"2023-07-06T01:20:25.886519+0000","last_clean":"2023-07-06T01:20:25.886519+0000","last_became_active":"2023-07-06T01:20:24.860803+0000","last_became_peered":"2023-07-06T01:20:24.860803+0000","last_unstale":"2023-07-06T01:20:25.886519+0000","last_undegraded":"2023-07-06T01:20:25.886519+0000","last_fullsized":"2023-07-06T01:20:25.886519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:17.181181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.830814+0000","last_change":"2023-07-06T01:20:22.830814+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.830814+0000","last_undegraded":"2023-07-06T01:20:22.830814+0000","last_fullsized":"2023-07-06T01:20:22.830814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862183+0000","last_change":"2023-07-06T01:20:24.861226+0000","last_active":"2023-07-06T01:20:24.862183+0000","last_peered":"2023-07-06T01:20:24.862183+0000","last_clean":"2023-07-06T01:20:24.862183+0000","last_became_active":"2023-07-06T01:20:24.860944+0000","last_became_peered":"2023-07-06T01:20:24.860944+0000","last_unstale":"2023-07-06T01:20:24.862183+0000","last_undegraded":"2023-07-06T01:20:24.862183+0000","last_fullsized":"2023-07-06T01:20:24.862183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:12.103457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884125+0000","last_change":"2023-07-06T01:20:23.849860+0000","last_active":"2023-07-06T01:20:25.884125+0000","last_peered":"2023-07-06T01:20:25.884125+0000","last_clean":"2023-07-06T01:20:25.884125+0000","last_became_active":"2023-07-06T01:20:23.849507+0000","last_became_peered":"2023-07-06T01:20:23.849507+0000","last_unstale":"2023-07-06T01:20:25.884125+0000","last_undegraded":"2023-07-06T01:20:25.884125+0000","last_fullsized":"2023-07-06T01:20:25.884125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:51.413216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.826768+0000","last_change":"2023-07-06T01:20:22.826768+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.826768+0000","last_undegraded":"2023-07-06T01:20:22.826768+0000","last_fullsized":"2023-07-06T01:20:22.826768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T01:20:22.827804+0000","last_change":"2023-07-06T01:19:20.074195+0000","last_active":"2023-07-06T01:20:22.827804+0000","last_peered":"2023-07-06T01:20:22.827804+0000","last_clean":"2023-07-06T01:20:22.827804+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:20:22.827804+0000","last_undegraded":"2023-07-06T01:20:22.827804+0000","last_fullsized":"2023-07-06T01:20:22.827804+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:38:31.352462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.842093+0000","last_change":"2023-07-06T01:20:23.849570+0000","last_active":"2023-07-06T01:20:24.842093+0000","last_peered":"2023-07-06T01:20:24.842093+0000","last_clean":"2023-07-06T01:20:24.842093+0000","last_became_active":"2023-07-06T01:20:23.849255+0000","last_became_peered":"2023-07-06T01:20:23.849255+0000","last_unstale":"2023-07-06T01:20:24.842093+0000","last_undegraded":"2023-07-06T01:20:24.842093+0000","last_fullsized":"2023-07-06T01:20:24.842093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:53:56.761538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.846124+0000","last_change":"2023-07-06T01:19:19.587110+0000","last_active":"2023-07-06T01:20:24.846124+0000","last_peered":"2023-07-06T01:20:24.846124+0000","last_clean":"2023-07-06T01:20:24.846124+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:20:24.846124+0000","last_undegraded":"2023-07-06T01:20:24.846124+0000","last_fullsized":"2023-07-06T01:20:24.846124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:49:50.820241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859472+0000","last_change":"2023-07-06T01:20:24.858252+0000","last_active":"2023-07-06T01:20:24.859472+0000","last_peered":"2023-07-06T01:20:24.859472+0000","last_clean":"2023-07-06T01:20:24.859472+0000","last_became_active":"2023-07-06T01:20:24.856691+0000","last_became_peered":"2023-07-06T01:20:24.856691+0000","last_unstale":"2023-07-06T01:20:24.859472+0000","last_undegraded":"2023-07-06T01:20:24.859472+0000","last_fullsized":"2023-07-06T01:20:24.859472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:03.093141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887051+0000","last_change":"2023-07-06T01:20:23.847756+0000","last_active":"2023-07-06T01:20:25.887051+0000","last_peered":"2023-07-06T01:20:25.887051+0000","last_clean":"2023-07-06T01:20:25.887051+0000","last_became_active":"2023-07-06T01:20:23.847392+0000","last_became_peered":"2023-07-06T01:20:23.847392+0000","last_unstale":"2023-07-06T01:20:25.887051+0000","last_undegraded":"2023-07-06T01:20:25.887051+0000","last_fullsized":"2023-07-06T01:20:25.887051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:50:55.073152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":35,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836206+0000","last_change":"2023-07-06T01:19:19.584242+0000","last_active":"2023-07-06T01:20:24.836206+0000","last_peered":"2023-07-06T01:20:24.836206+0000","last_clean":"2023-07-06T01:20:24.836206+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:20:24.836206+0000","last_undegraded":"2023-07-06T01:20:24.836206+0000","last_fullsized":"2023-07-06T01:20:24.836206+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:02:18.883377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":183,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T01:20:22.831308+0000","last_change":"2023-07-06T01:19:19.586495+0000","last_active":"2023-07-06T01:20:22.831308+0000","last_peered":"2023-07-06T01:20:22.831308+0000","last_clean":"2023-07-06T01:20:22.831308+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:20:22.831308+0000","last_undegraded":"2023-07-06T01:20:22.831308+0000","last_fullsized":"2023-07-06T01:20:22.831308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:06:26.958597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.885927+0000","last_change":"2023-07-06T01:20:23.854385+0000","last_active":"2023-07-06T01:20:25.885927+0000","last_peered":"2023-07-06T01:20:25.885927+0000","last_clean":"2023-07-06T01:20:25.885927+0000","last_became_active":"2023-07-06T01:20:23.853982+0000","last_became_peered":"2023-07-06T01:20:23.853982+0000","last_unstale":"2023-07-06T01:20:25.885927+0000","last_undegraded":"2023-07-06T01:20:25.885927+0000","last_fullsized":"2023-07-06T01:20:25.885927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:01.772125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T01:20:22.831236+0000","last_change":"2023-07-06T01:19:19.586426+0000","last_active":"2023-07-06T01:20:22.831236+0000","last_peered":"2023-07-06T01:20:22.831236+0000","last_clean":"2023-07-06T01:20:22.831236+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:20:22.831236+0000","last_undegraded":"2023-07-06T01:20:22.831236+0000","last_fullsized":"2023-07-06T01:20:22.831236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:46.933540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886966+0000","last_change":"2023-07-06T01:20:24.855685+0000","last_active":"2023-07-06T01:20:25.886966+0000","last_peered":"2023-07-06T01:20:25.886966+0000","last_clean":"2023-07-06T01:20:25.886966+0000","last_became_active":"2023-07-06T01:20:24.855347+0000","last_became_peered":"2023-07-06T01:20:24.855347+0000","last_unstale":"2023-07-06T01:20:25.886966+0000","last_undegraded":"2023-07-06T01:20:25.886966+0000","last_fullsized":"2023-07-06T01:20:25.886966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:39:36.610034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.830379+0000","last_change":"2023-07-06T01:20:22.830379+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.830379+0000","last_undegraded":"2023-07-06T01:20:22.830379+0000","last_fullsized":"2023-07-06T01:20:22.830379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T01:20:22.831212+0000","last_change":"2023-07-06T01:19:19.586626+0000","last_active":"2023-07-06T01:20:22.831212+0000","last_peered":"2023-07-06T01:20:22.831212+0000","last_clean":"2023-07-06T01:20:22.831212+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:20:22.831212+0000","last_undegraded":"2023-07-06T01:20:22.831212+0000","last_fullsized":"2023-07-06T01:20:22.831212+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:06:51.308527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.852493+0000","last_change":"2023-07-06T01:20:24.845956+0000","last_active":"2023-07-06T01:20:24.852493+0000","last_peered":"2023-07-06T01:20:24.852493+0000","last_clean":"2023-07-06T01:20:24.852493+0000","last_became_active":"2023-07-06T01:20:24.845698+0000","last_became_peered":"2023-07-06T01:20:24.845698+0000","last_unstale":"2023-07-06T01:20:24.852493+0000","last_undegraded":"2023-07-06T01:20:24.852493+0000","last_fullsized":"2023-07-06T01:20:24.852493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:41.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834898+0000","last_change":"2023-07-06T01:20:23.847902+0000","last_active":"2023-07-06T01:20:25.834898+0000","last_peered":"2023-07-06T01:20:25.834898+0000","last_clean":"2023-07-06T01:20:25.834898+0000","last_became_active":"2023-07-06T01:20:23.847543+0000","last_became_peered":"2023-07-06T01:20:23.847543+0000","last_unstale":"2023-07-06T01:20:25.834898+0000","last_undegraded":"2023-07-06T01:20:25.834898+0000","last_fullsized":"2023-07-06T01:20:25.834898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:51.462337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886549+0000","last_change":"2023-07-06T01:19:19.584086+0000","last_active":"2023-07-06T01:20:25.886549+0000","last_peered":"2023-07-06T01:20:25.886549+0000","last_clean":"2023-07-06T01:20:25.886549+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:20:25.886549+0000","last_undegraded":"2023-07-06T01:20:25.886549+0000","last_fullsized":"2023-07-06T01:20:25.886549+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:01:33.579869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855022+0000","last_change":"2023-07-06T01:20:24.851662+0000","last_active":"2023-07-06T01:20:24.855022+0000","last_peered":"2023-07-06T01:20:24.855022+0000","last_clean":"2023-07-06T01:20:24.855022+0000","last_became_active":"2023-07-06T01:20:24.851349+0000","last_became_peered":"2023-07-06T01:20:24.851349+0000","last_unstale":"2023-07-06T01:20:24.855022+0000","last_undegraded":"2023-07-06T01:20:24.855022+0000","last_fullsized":"2023-07-06T01:20:24.855022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:50.421447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836073+0000","last_change":"2023-07-06T01:20:23.845197+0000","last_active":"2023-07-06T01:20:25.836073+0000","last_peered":"2023-07-06T01:20:25.836073+0000","last_clean":"2023-07-06T01:20:25.836073+0000","last_became_active":"2023-07-06T01:20:23.844508+0000","last_became_peered":"2023-07-06T01:20:23.844508+0000","last_unstale":"2023-07-06T01:20:25.836073+0000","last_undegraded":"2023-07-06T01:20:25.836073+0000","last_fullsized":"2023-07-06T01:20:25.836073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:00.176807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836633+0000","last_change":"2023-07-06T01:19:19.586424+0000","last_active":"2023-07-06T01:20:25.836633+0000","last_peered":"2023-07-06T01:20:25.836633+0000","last_clean":"2023-07-06T01:20:25.836633+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:20:25.836633+0000","last_undegraded":"2023-07-06T01:20:25.836633+0000","last_fullsized":"2023-07-06T01:20:25.836633+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:04:57.410877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834442+0000","last_change":"2023-07-06T01:20:24.849984+0000","last_active":"2023-07-06T01:20:25.834442+0000","last_peered":"2023-07-06T01:20:25.834442+0000","last_clean":"2023-07-06T01:20:25.834442+0000","last_became_active":"2023-07-06T01:20:24.849603+0000","last_became_peered":"2023-07-06T01:20:24.849603+0000","last_unstale":"2023-07-06T01:20:25.834442+0000","last_undegraded":"2023-07-06T01:20:25.834442+0000","last_fullsized":"2023-07-06T01:20:25.834442+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:39:50.052979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834826+0000","last_change":"2023-07-06T01:20:23.842452+0000","last_active":"2023-07-06T01:20:25.834826+0000","last_peered":"2023-07-06T01:20:25.834826+0000","last_clean":"2023-07-06T01:20:25.834826+0000","last_became_active":"2023-07-06T01:20:23.842176+0000","last_became_peered":"2023-07-06T01:20:23.842176+0000","last_unstale":"2023-07-06T01:20:25.834826+0000","last_undegraded":"2023-07-06T01:20:25.834826+0000","last_fullsized":"2023-07-06T01:20:25.834826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:16.176519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836914+0000","last_change":"2023-07-06T01:19:19.583699+0000","last_active":"2023-07-06T01:20:25.836914+0000","last_peered":"2023-07-06T01:20:25.836914+0000","last_clean":"2023-07-06T01:20:25.836914+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:20:25.836914+0000","last_undegraded":"2023-07-06T01:20:25.836914+0000","last_fullsized":"2023-07-06T01:20:25.836914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:52:09.171519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":94,"acting":94,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":721739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.505},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.439},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45200000000000001}]},{"osd":3,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53200000000000003}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.34100000000000003}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.375}]}]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":344064,"data_stored":137541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.499},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47599999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.436},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.39800000000000002}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.622}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":966656,"data_stored":741755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":12.23,"5min":12.23,"15min":12.23},"last":0.311},{"interface":"front","average":{"1min":1.839,"5min":1.839,"15min":1.839},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":15.208,"5min":15.208,"15min":15.208},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":12.074,"5min":12.074,"15min":12.074},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":11.988,"5min":11.988,"15min":11.988},"last":0.73999999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.173,"5min":2.173,"15min":2.173},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"last":0.442},{"interface":"front","average":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":15.244999999999999,"5min":15.244999999999999,"15min":15.244999999999999},"last":0.379}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":11.923,"5min":11.923,"15min":11.923},"last":2.6080000000000001},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":12.019,"5min":12.019,"15min":12.019},"last":3.5950000000000002}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":11.999000000000001,"5min":11.999000000000001,"15min":11.999000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":11.951000000000001,"5min":11.951000000000001,"15min":11.951000000000001},"last":0.624}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":15.167999999999999,"5min":15.167999999999999,"15min":15.167999999999999},"last":2.7490000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":12.041,"5min":12.041,"15min":12.041},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":15.115,"5min":15.115,"15min":15.115},"last":1.625},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":11.965,"5min":11.965,"15min":11.965},"last":1.8200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":157984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":0.251}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.215},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.27200000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.503},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":6.3049999999999997,"5min":6.3049999999999997,"15min":6.3049999999999997},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.2430000000000003,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.63900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.436},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.625}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54900000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":152895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.307},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.375}]},{"osd":2,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.32700000000000001}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":3.8130000000000002,"5min":3.8130000000000002,"15min":3.8130000000000002},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.71899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":158062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.5},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.436}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":158532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.629},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.505},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.439}]},{"osd":5,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42399999999999999}]},{"osd":7,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:20:29.009 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:20:29.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.163+0000 7f854f719700 1 Processor -- start 2023-07-06T01:20:29.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.164+0000 7f854f719700 1 -- start start 2023-07-06T01:20:29.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.164+0000 7f854f719700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f854806b320 0x7f854806c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.164+0000 7f854f719700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8548071f30 con 0x7f854815a610 2023-07-06T01:20:29.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.164+0000 7f854f719700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854806b770 con 0x7f854806c730 2023-07-06T01:20:29.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.164+0000 7f854f719700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854815a9e0 con 0x7f854806b320 2023-07-06T01:20:29.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f854d4b5700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f854815a610 0x7f854815eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38174/0 (socket says 172.21.15.22:38174) 2023-07-06T01:20:29.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f854d4b5700 1 -- 172.21.15.22:0/4221221298 learned_addr learned my addr 172.21.15.22:0/4221221298 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:29.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f854ccb4700 1 --2- 172.21.15.22:0/4221221298 >> v2:172.21.15.103:3300/0 conn(0x7f854806b320 0x7f854806c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135711666 0 0) 0x7f8548071f30 con 0x7f854815a610 2023-07-06T01:20:29.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85340035b0 con 0x7f854815a610 2023-07-06T01:20:29.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4229279425 0 0) 0x7f85340035b0 con 0x7f854815a610 2023-07-06T01:20:29.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 >> v2:172.21.15.103:3300/0 conn(0x7f854806b320 msgr2=0x7f854806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:29.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 --2- 172.21.15.22:0/4221221298 >> v2:172.21.15.103:3300/0 conn(0x7f854806b320 0x7f854806c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 >> v1:172.21.15.103:6789/0 conn(0x7f854806c730 legacy=0x7f85480c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8548071320 con 0x7f854815a610 2023-07-06T01:20:29.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f8544001670 con 0x7f854815a610 2023-07-06T01:20:29.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 >> v1:172.21.15.22:6789/0 conn(0x7f854815a610 legacy=0x7f854815eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 --2- 172.21.15.22:0/4221221298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8534003a00 0x7f8534003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 --2- 172.21.15.22:0/4221221298 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8534004850 0x7f8534004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 --2- 172.21.15.22:0/4221221298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8534005200 0x7f853400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854815a9e0 con 0x7f8534003a00 2023-07-06T01:20:29.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853400c080 con 0x7f8534005200 2023-07-06T01:20:29.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853400c240 con 0x7f8534004850 2023-07-06T01:20:29.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f854d4b5700 1 --2- 172.21.15.22:0/4221221298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8534003a00 0x7f8534003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f854dcb6700 1 --2- 172.21.15.22:0/4221221298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8534005200 0x7f853400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.165+0000 7f854ccb4700 1 --2- 172.21.15.22:0/4221221298 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8534004850 0x7f8534004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f854d4b5700 1 -- 172.21.15.22:0/4221221298 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8534004850 msgr2=0x7f8534004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:29.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f854d4b5700 1 --2- 172.21.15.22:0/4221221298 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8534004850 0x7f8534004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f854d4b5700 1 -- 172.21.15.22:0/4221221298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8534005200 msgr2=0x7f853400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:29.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f854d4b5700 1 --2- 172.21.15.22:0/4221221298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8534005200 0x7f853400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f854d4b5700 1 -- 172.21.15.22:0/4221221298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85340035b0 con 0x7f8534003a00 2023-07-06T01:20:29.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f854dcb6700 1 -- 172.21.15.22:0/4221221298 reap_dead start 2023-07-06T01:20:29.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f854d4b5700 1 --2- 172.21.15.22:0/4221221298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8534003a00 0x7f8534003e10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f854400c240 tx=0x7f8534005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f713389441862800 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:29.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544007c10 con 0x7f8534003a00 2023-07-06T01:20:29.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854400ce00 con 0x7f8534003a00 2023-07-06T01:20:29.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f853bfff700 1 -- 172.21.15.22:0/4221221298 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544005ba0 con 0x7f8534003a00 2023-07-06T01:20:29.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f854f719700 1 -- 172.21.15.22:0/4221221298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8534003a00 msgr2=0x7f8534003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f854f719700 1 --2- 172.21.15.22:0/4221221298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8534003a00 0x7f8534003e10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f854400c240 tx=0x7f8534005880 comp rx=0 tx=0).stop 2023-07-06T01:20:29.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f854f719700 1 -- 172.21.15.22:0/4221221298 shutdown_connections 2023-07-06T01:20:29.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f854f719700 1 --2- 172.21.15.22:0/4221221298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8534003a00 0x7f8534003e10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f854f719700 1 -- 172.21.15.22:0/4221221298 >> 172.21.15.22:0/4221221298 conn(0x7f85480ccb00 msgr2=0x7f85480cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:29.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.166+0000 7f854f719700 1 -- 172.21.15.22:0/4221221298 shutdown_connections 2023-07-06T01:20:29.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.167+0000 7f854f719700 1 -- 172.21.15.22:0/4221221298 wait complete. 2023-07-06T01:20:29.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.168+0000 7f854f719700 1 Processor -- start 2023-07-06T01:20:29.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.168+0000 7f854f719700 1 -- start start 2023-07-06T01:20:29.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.168+0000 7f854f719700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8534003a00 0x7f85480b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.168+0000 7f854f719700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85480cebb0 0x7f8548154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.168+0000 7f854f719700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85481553f0 0x7f854815c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.168+0000 7f854f719700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85480ae2c0 con 0x7f85480cebb0 2023-07-06T01:20:29.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.168+0000 7f854f719700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85480cf020 con 0x7f8534003a00 2023-07-06T01:20:29.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.168+0000 7f854f719700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85480ccfc0 con 0x7f85481553f0 2023-07-06T01:20:29.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854ccb4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85480cebb0 0x7f8548154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854ccb4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85480cebb0 0x7f8548154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47444/0 (socket says 172.21.15.22:47444) 2023-07-06T01:20:29.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854ccb4700 1 -- 172.21.15.22:0/4052266845 learned_addr learned my addr 172.21.15.22:0/4052266845 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854d4b5700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8534003a00 0x7f85480b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854dcb6700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85481553f0 0x7f854815c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854ccb4700 1 -- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85481553f0 msgr2=0x7f854815c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854ccb4700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85481553f0 0x7f854815c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854ccb4700 1 -- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8534003a00 msgr2=0x7f85480b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854ccb4700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8534003a00 0x7f85480b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854ccb4700 1 -- 172.21.15.22:0/4052266845 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85480b2800 con 0x7f85480cebb0 2023-07-06T01:20:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854d4b5700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8534003a00 0x7f85480b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854dcb6700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85481553f0 0x7f854815c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854ccb4700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85480cebb0 0x7f8548154660 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f853c011720 tx=0x7f853c011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f8539ffb700 1 -- 172.21.15.22:0/4052266845 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c015870 con 0x7f85480cebb0 2023-07-06T01:20:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f8539ffb700 1 -- 172.21.15.22:0/4052266845 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853c0159d0 con 0x7f85480cebb0 2023-07-06T01:20:29.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854f719700 1 -- 172.21.15.22:0/4052266845 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85480b2ae0 con 0x7f85480cebb0 2023-07-06T01:20:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f8539ffb700 1 -- 172.21.15.22:0/4052266845 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c015d40 con 0x7f85480cebb0 2023-07-06T01:20:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.169+0000 7f854f719700 1 -- 172.21.15.22:0/4052266845 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85480cd210 con 0x7f85480cebb0 2023-07-06T01:20:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.170+0000 7f854f719700 1 -- 172.21.15.22:0/4052266845 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85480cd580 con 0x7f85480cebb0 2023-07-06T01:20:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.171+0000 7f8539ffb700 1 -- 172.21.15.22:0/4052266845 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f853c01d400 con 0x7f85480cebb0 2023-07-06T01:20:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.171+0000 7f8539ffb700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f852c06ace0 0x7f852c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.171+0000 7f8539ffb700 1 -- 172.21.15.22:0/4052266845 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f853c089b60 con 0x7f85480cebb0 2023-07-06T01:20:29.181 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.172+0000 7f854d4b5700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f852c06ace0 0x7f852c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.172+0000 7f854d4b5700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f852c06ace0 0x7f852c06d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8544001930 tx=0x7f8544007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:29.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.176+0000 7f8539ffb700 1 -- 172.21.15.22:0/4052266845 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f853c054ca0 con 0x7f85480cebb0 2023-07-06T01:20:29.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.367+0000 7f854f719700 1 -- 172.21.15.22:0/4052266845 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85480718e0 con 0x7f852c06ace0 2023-07-06T01:20:29.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.376+0000 7f8539ffb700 1 -- 172.21.15.22:0/4052266845 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308073 (secure 0 0 0) 0x7f8544007bb0 con 0x7f852c06ace0 2023-07-06T01:20:29.380 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:29.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.379+0000 7f854f719700 1 -- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f852c06ace0 msgr2=0x7f852c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.379+0000 7f854f719700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f852c06ace0 0x7f852c06d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8544001930 tx=0x7f8544007000 comp rx=0 tx=0).stop 2023-07-06T01:20:29.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.379+0000 7f854f719700 1 -- 172.21.15.22:0/4052266845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85480cebb0 msgr2=0x7f8548154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.382 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.379+0000 7f854f719700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85480cebb0 0x7f8548154660 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f853c011720 tx=0x7f853c011ae0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.379+0000 7f854f719700 1 -- 172.21.15.22:0/4052266845 shutdown_connections 2023-07-06T01:20:29.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.379+0000 7f854f719700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85481553f0 0x7f854815c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.379+0000 7f854f719700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f852c06ace0 0x7f852c06d190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.379+0000 7f854f719700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8534003a00 0x7f85480b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.379+0000 7f854f719700 1 --2- 172.21.15.22:0/4052266845 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85480cebb0 0x7f8548154660 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.379+0000 7f854f719700 1 -- 172.21.15.22:0/4052266845 >> 172.21.15.22:0/4052266845 conn(0x7f85480ccb00 msgr2=0x7f854815ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:29.383 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.379+0000 7f854f719700 1 -- 172.21.15.22:0/4052266845 shutdown_connections 2023-07-06T01:20:29.384 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.379+0000 7f854f719700 1 -- 172.21.15.22:0/4052266845 wait complete. 2023-07-06T01:20:29.384 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:20:29.393 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-07-06T01:20:28.014418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":186,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219684,"kb_used_data":4068,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755036,"statfs":{"total":773094113280,"available":772869156864,"internally_reserved":0,"allocated":4165632,"data_stored":2386528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.189545"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855626+0000","last_change":"2023-07-06T01:20:24.852545+0000","last_active":"2023-07-06T01:20:24.855626+0000","last_peered":"2023-07-06T01:20:24.855626+0000","last_clean":"2023-07-06T01:20:24.855626+0000","last_became_active":"2023-07-06T01:20:24.852249+0000","last_became_peered":"2023-07-06T01:20:24.852249+0000","last_unstale":"2023-07-06T01:20:24.855626+0000","last_undegraded":"2023-07-06T01:20:24.855626+0000","last_fullsized":"2023-07-06T01:20:24.855626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:17.181181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.854187+0000","last_change":"2023-07-06T01:20:24.850725+0000","last_active":"2023-07-06T01:20:24.854187+0000","last_peered":"2023-07-06T01:20:24.854187+0000","last_clean":"2023-07-06T01:20:24.854187+0000","last_became_active":"2023-07-06T01:20:24.849359+0000","last_became_peered":"2023-07-06T01:20:24.849359+0000","last_unstale":"2023-07-06T01:20:24.854187+0000","last_undegraded":"2023-07-06T01:20:24.854187+0000","last_fullsized":"2023-07-06T01:20:24.854187+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:20.409326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834789+0000","last_change":"2023-07-06T01:20:24.855847+0000","last_active":"2023-07-06T01:20:25.834789+0000","last_peered":"2023-07-06T01:20:25.834789+0000","last_clean":"2023-07-06T01:20:25.834789+0000","last_became_active":"2023-07-06T01:20:24.855571+0000","last_became_peered":"2023-07-06T01:20:24.855571+0000","last_unstale":"2023-07-06T01:20:25.834789+0000","last_undegraded":"2023-07-06T01:20:25.834789+0000","last_fullsized":"2023-07-06T01:20:25.834789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884281+0000","last_change":"2023-07-06T01:20:24.860708+0000","last_active":"2023-07-06T01:20:25.884281+0000","last_peered":"2023-07-06T01:20:25.884281+0000","last_clean":"2023-07-06T01:20:25.884281+0000","last_became_active":"2023-07-06T01:20:24.860220+0000","last_became_peered":"2023-07-06T01:20:24.860220+0000","last_unstale":"2023-07-06T01:20:25.884281+0000","last_undegraded":"2023-07-06T01:20:25.884281+0000","last_fullsized":"2023-07-06T01:20:25.884281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:42:33.239141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884127+0000","last_change":"2023-07-06T01:20:24.850342+0000","last_active":"2023-07-06T01:20:25.884127+0000","last_peered":"2023-07-06T01:20:25.884127+0000","last_clean":"2023-07-06T01:20:25.884127+0000","last_became_active":"2023-07-06T01:20:24.850083+0000","last_became_peered":"2023-07-06T01:20:24.850083+0000","last_unstale":"2023-07-06T01:20:25.884127+0000","last_undegraded":"2023-07-06T01:20:25.884127+0000","last_fullsized":"2023-07-06T01:20:25.884127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:46:35.434349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.850816+0000","last_change":"2023-07-06T01:20:24.848000+0000","last_active":"2023-07-06T01:20:24.850816+0000","last_peered":"2023-07-06T01:20:24.850816+0000","last_clean":"2023-07-06T01:20:24.850816+0000","last_became_active":"2023-07-06T01:20:24.847814+0000","last_became_peered":"2023-07-06T01:20:24.847814+0000","last_unstale":"2023-07-06T01:20:24.850816+0000","last_undegraded":"2023-07-06T01:20:24.850816+0000","last_fullsized":"2023-07-06T01:20:24.850816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:51.766923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886546+0000","last_change":"2023-07-06T01:20:24.856730+0000","last_active":"2023-07-06T01:20:25.886546+0000","last_peered":"2023-07-06T01:20:25.886546+0000","last_clean":"2023-07-06T01:20:25.886546+0000","last_became_active":"2023-07-06T01:20:24.856460+0000","last_became_peered":"2023-07-06T01:20:24.856460+0000","last_unstale":"2023-07-06T01:20:25.886546+0000","last_undegraded":"2023-07-06T01:20:25.886546+0000","last_fullsized":"2023-07-06T01:20:25.886546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:52.466998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859689+0000","last_change":"2023-07-06T01:20:24.858297+0000","last_active":"2023-07-06T01:20:24.859689+0000","last_peered":"2023-07-06T01:20:24.859689+0000","last_clean":"2023-07-06T01:20:24.859689+0000","last_became_active":"2023-07-06T01:20:24.858033+0000","last_became_peered":"2023-07-06T01:20:24.858033+0000","last_unstale":"2023-07-06T01:20:24.859689+0000","last_undegraded":"2023-07-06T01:20:24.859689+0000","last_fullsized":"2023-07-06T01:20:24.859689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:38.837442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834859+0000","last_change":"2023-07-06T01:20:24.859281+0000","last_active":"2023-07-06T01:20:25.834859+0000","last_peered":"2023-07-06T01:20:25.834859+0000","last_clean":"2023-07-06T01:20:25.834859+0000","last_became_active":"2023-07-06T01:20:24.858982+0000","last_became_peered":"2023-07-06T01:20:24.858982+0000","last_unstale":"2023-07-06T01:20:25.834859+0000","last_undegraded":"2023-07-06T01:20:25.834859+0000","last_fullsized":"2023-07-06T01:20:25.834859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:41.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886707+0000","last_change":"2023-07-06T01:20:24.854480+0000","last_active":"2023-07-06T01:20:25.886707+0000","last_peered":"2023-07-06T01:20:25.886707+0000","last_clean":"2023-07-06T01:20:25.886707+0000","last_became_active":"2023-07-06T01:20:24.854116+0000","last_became_peered":"2023-07-06T01:20:24.854116+0000","last_unstale":"2023-07-06T01:20:25.886707+0000","last_undegraded":"2023-07-06T01:20:25.886707+0000","last_fullsized":"2023-07-06T01:20:25.886707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:12.103457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886974+0000","last_change":"2023-07-06T01:20:24.854685+0000","last_active":"2023-07-06T01:20:25.886974+0000","last_peered":"2023-07-06T01:20:25.886974+0000","last_clean":"2023-07-06T01:20:25.886974+0000","last_became_active":"2023-07-06T01:20:24.853605+0000","last_became_peered":"2023-07-06T01:20:24.853605+0000","last_unstale":"2023-07-06T01:20:25.886974+0000","last_undegraded":"2023-07-06T01:20:25.886974+0000","last_fullsized":"2023-07-06T01:20:25.886974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:54:12.127760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834625+0000","last_change":"2023-07-06T01:20:24.853728+0000","last_active":"2023-07-06T01:20:25.834625+0000","last_peered":"2023-07-06T01:20:25.834625+0000","last_clean":"2023-07-06T01:20:25.834625+0000","last_became_active":"2023-07-06T01:20:24.853427+0000","last_became_peered":"2023-07-06T01:20:24.853427+0000","last_unstale":"2023-07-06T01:20:25.834625+0000","last_undegraded":"2023-07-06T01:20:25.834625+0000","last_fullsized":"2023-07-06T01:20:25.834625+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:31.557538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886831+0000","last_change":"2023-07-06T01:20:24.856012+0000","last_active":"2023-07-06T01:20:25.886831+0000","last_peered":"2023-07-06T01:20:25.886831+0000","last_clean":"2023-07-06T01:20:25.886831+0000","last_became_active":"2023-07-06T01:20:24.855762+0000","last_became_peered":"2023-07-06T01:20:24.855762+0000","last_unstale":"2023-07-06T01:20:25.886831+0000","last_undegraded":"2023-07-06T01:20:25.886831+0000","last_fullsized":"2023-07-06T01:20:25.886831+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:45.220808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886397+0000","last_change":"2023-07-06T01:20:24.857423+0000","last_active":"2023-07-06T01:20:25.886397+0000","last_peered":"2023-07-06T01:20:25.886397+0000","last_clean":"2023-07-06T01:20:25.886397+0000","last_became_active":"2023-07-06T01:20:24.857164+0000","last_became_peered":"2023-07-06T01:20:24.857164+0000","last_unstale":"2023-07-06T01:20:25.886397+0000","last_undegraded":"2023-07-06T01:20:25.886397+0000","last_fullsized":"2023-07-06T01:20:25.886397+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836700+0000","last_change":"2023-07-06T01:20:24.860237+0000","last_active":"2023-07-06T01:20:25.836700+0000","last_peered":"2023-07-06T01:20:25.836700+0000","last_clean":"2023-07-06T01:20:25.836700+0000","last_became_active":"2023-07-06T01:20:24.859324+0000","last_became_peered":"2023-07-06T01:20:24.859324+0000","last_unstale":"2023-07-06T01:20:25.836700+0000","last_undegraded":"2023-07-06T01:20:25.836700+0000","last_fullsized":"2023-07-06T01:20:25.836700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:50.731526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884196+0000","last_change":"2023-07-06T01:20:24.852638+0000","last_active":"2023-07-06T01:20:25.884196+0000","last_peered":"2023-07-06T01:20:25.884196+0000","last_clean":"2023-07-06T01:20:25.884196+0000","last_became_active":"2023-07-06T01:20:24.852341+0000","last_became_peered":"2023-07-06T01:20:24.852341+0000","last_unstale":"2023-07-06T01:20:25.884196+0000","last_undegraded":"2023-07-06T01:20:25.884196+0000","last_fullsized":"2023-07-06T01:20:25.884196+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:17:34.632929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836872+0000","last_change":"2023-07-06T01:20:24.859626+0000","last_active":"2023-07-06T01:20:25.836872+0000","last_peered":"2023-07-06T01:20:25.836872+0000","last_clean":"2023-07-06T01:20:25.836872+0000","last_became_active":"2023-07-06T01:20:24.859274+0000","last_became_peered":"2023-07-06T01:20:24.859274+0000","last_unstale":"2023-07-06T01:20:25.836872+0000","last_undegraded":"2023-07-06T01:20:25.836872+0000","last_fullsized":"2023-07-06T01:20:25.836872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:03:24.140950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886696+0000","last_change":"2023-07-06T01:20:24.854443+0000","last_active":"2023-07-06T01:20:25.886696+0000","last_peered":"2023-07-06T01:20:25.886696+0000","last_clean":"2023-07-06T01:20:25.886696+0000","last_became_active":"2023-07-06T01:20:24.853256+0000","last_became_peered":"2023-07-06T01:20:24.853256+0000","last_unstale":"2023-07-06T01:20:25.886696+0000","last_undegraded":"2023-07-06T01:20:25.886696+0000","last_fullsized":"2023-07-06T01:20:25.886696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:20.409326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884398+0000","last_change":"2023-07-06T01:20:24.852693+0000","last_active":"2023-07-06T01:20:25.884398+0000","last_peered":"2023-07-06T01:20:25.884398+0000","last_clean":"2023-07-06T01:20:25.884398+0000","last_became_active":"2023-07-06T01:20:24.852395+0000","last_became_peered":"2023-07-06T01:20:24.852395+0000","last_unstale":"2023-07-06T01:20:25.884398+0000","last_undegraded":"2023-07-06T01:20:25.884398+0000","last_fullsized":"2023-07-06T01:20:25.884398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:35.585237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836707+0000","last_change":"2023-07-06T01:20:24.856089+0000","last_active":"2023-07-06T01:20:25.836707+0000","last_peered":"2023-07-06T01:20:25.836707+0000","last_clean":"2023-07-06T01:20:25.836707+0000","last_became_active":"2023-07-06T01:20:24.855834+0000","last_became_peered":"2023-07-06T01:20:24.855834+0000","last_unstale":"2023-07-06T01:20:25.836707+0000","last_undegraded":"2023-07-06T01:20:25.836707+0000","last_fullsized":"2023-07-06T01:20:25.836707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:03.093141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859016+0000","last_change":"2023-07-06T01:20:24.856754+0000","last_active":"2023-07-06T01:20:24.859016+0000","last_peered":"2023-07-06T01:20:24.859016+0000","last_clean":"2023-07-06T01:20:24.859016+0000","last_became_active":"2023-07-06T01:20:24.856396+0000","last_became_peered":"2023-07-06T01:20:24.856396+0000","last_unstale":"2023-07-06T01:20:24.859016+0000","last_undegraded":"2023-07-06T01:20:24.859016+0000","last_fullsized":"2023-07-06T01:20:24.859016+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:06:02.668449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835952+0000","last_change":"2023-07-06T01:20:24.854996+0000","last_active":"2023-07-06T01:20:25.835952+0000","last_peered":"2023-07-06T01:20:25.835952+0000","last_clean":"2023-07-06T01:20:25.835952+0000","last_became_active":"2023-07-06T01:20:24.854737+0000","last_became_peered":"2023-07-06T01:20:24.854737+0000","last_unstale":"2023-07-06T01:20:25.835952+0000","last_undegraded":"2023-07-06T01:20:25.835952+0000","last_fullsized":"2023-07-06T01:20:25.835952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:51.766923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886641+0000","last_change":"2023-07-06T01:20:24.856804+0000","last_active":"2023-07-06T01:20:25.886641+0000","last_peered":"2023-07-06T01:20:25.886641+0000","last_clean":"2023-07-06T01:20:25.886641+0000","last_became_active":"2023-07-06T01:20:24.856490+0000","last_became_peered":"2023-07-06T01:20:24.856490+0000","last_unstale":"2023-07-06T01:20:25.886641+0000","last_undegraded":"2023-07-06T01:20:25.886641+0000","last_fullsized":"2023-07-06T01:20:25.886641+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:23:07.815642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835630+0000","last_change":"2023-07-06T01:20:24.850799+0000","last_active":"2023-07-06T01:20:25.835630+0000","last_peered":"2023-07-06T01:20:25.835630+0000","last_clean":"2023-07-06T01:20:25.835630+0000","last_became_active":"2023-07-06T01:20:24.850463+0000","last_became_peered":"2023-07-06T01:20:24.850463+0000","last_unstale":"2023-07-06T01:20:25.835630+0000","last_undegraded":"2023-07-06T01:20:25.835630+0000","last_fullsized":"2023-07-06T01:20:25.835630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:50.421447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835944+0000","last_change":"2023-07-06T01:20:24.857969+0000","last_active":"2023-07-06T01:20:25.835944+0000","last_peered":"2023-07-06T01:20:25.835944+0000","last_clean":"2023-07-06T01:20:25.835944+0000","last_became_active":"2023-07-06T01:20:24.857674+0000","last_became_peered":"2023-07-06T01:20:24.857674+0000","last_unstale":"2023-07-06T01:20:25.835944+0000","last_undegraded":"2023-07-06T01:20:25.835944+0000","last_fullsized":"2023-07-06T01:20:25.835944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:57.090630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.856018+0000","last_change":"2023-07-06T01:20:24.854034+0000","last_active":"2023-07-06T01:20:24.856018+0000","last_peered":"2023-07-06T01:20:24.856018+0000","last_clean":"2023-07-06T01:20:24.856018+0000","last_became_active":"2023-07-06T01:20:24.853820+0000","last_became_peered":"2023-07-06T01:20:24.853820+0000","last_unstale":"2023-07-06T01:20:24.856018+0000","last_undegraded":"2023-07-06T01:20:24.856018+0000","last_fullsized":"2023-07-06T01:20:24.856018+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:57.090630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836884+0000","last_change":"2023-07-06T01:20:23.849203+0000","last_active":"2023-07-06T01:20:25.836884+0000","last_peered":"2023-07-06T01:20:25.836884+0000","last_clean":"2023-07-06T01:20:25.836884+0000","last_became_active":"2023-07-06T01:20:23.848883+0000","last_became_peered":"2023-07-06T01:20:23.848883+0000","last_unstale":"2023-07-06T01:20:25.836884+0000","last_undegraded":"2023-07-06T01:20:25.836884+0000","last_fullsized":"2023-07-06T01:20:25.836884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:06.770007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.860470+0000","last_change":"2023-07-06T01:20:24.859359+0000","last_active":"2023-07-06T01:20:24.860470+0000","last_peered":"2023-07-06T01:20:24.860470+0000","last_clean":"2023-07-06T01:20:24.860470+0000","last_became_active":"2023-07-06T01:20:24.859152+0000","last_became_peered":"2023-07-06T01:20:24.859152+0000","last_unstale":"2023-07-06T01:20:24.860470+0000","last_undegraded":"2023-07-06T01:20:24.860470+0000","last_fullsized":"2023-07-06T01:20:24.860470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:01.181469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834689+0000","last_change":"2023-07-06T01:20:23.845980+0000","last_active":"2023-07-06T01:20:25.834689+0000","last_peered":"2023-07-06T01:20:25.834689+0000","last_clean":"2023-07-06T01:20:25.834689+0000","last_became_active":"2023-07-06T01:20:23.845696+0000","last_became_peered":"2023-07-06T01:20:23.845696+0000","last_unstale":"2023-07-06T01:20:25.834689+0000","last_undegraded":"2023-07-06T01:20:25.834689+0000","last_fullsized":"2023-07-06T01:20:25.834689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:58.579906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.858947+0000","last_change":"2023-07-06T01:20:23.846742+0000","last_active":"2023-07-06T01:20:24.858947+0000","last_peered":"2023-07-06T01:20:24.858947+0000","last_clean":"2023-07-06T01:20:24.858947+0000","last_became_active":"2023-07-06T01:20:23.846440+0000","last_became_peered":"2023-07-06T01:20:23.846440+0000","last_unstale":"2023-07-06T01:20:24.858947+0000","last_undegraded":"2023-07-06T01:20:24.858947+0000","last_fullsized":"2023-07-06T01:20:24.858947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:06.770007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862194+0000","last_change":"2023-07-06T01:20:24.861558+0000","last_active":"2023-07-06T01:20:24.862194+0000","last_peered":"2023-07-06T01:20:24.862194+0000","last_clean":"2023-07-06T01:20:24.862194+0000","last_became_active":"2023-07-06T01:20:24.860857+0000","last_became_peered":"2023-07-06T01:20:24.860857+0000","last_unstale":"2023-07-06T01:20:24.862194+0000","last_undegraded":"2023-07-06T01:20:24.862194+0000","last_fullsized":"2023-07-06T01:20:24.862194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:52.466998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.827224+0000","last_change":"2023-07-06T01:20:22.827224+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.827224+0000","last_undegraded":"2023-07-06T01:20:22.827224+0000","last_fullsized":"2023-07-06T01:20:22.827224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834373+0000","last_change":"2023-07-06T01:20:24.839527+0000","last_active":"2023-07-06T01:20:25.834373+0000","last_peered":"2023-07-06T01:20:25.834373+0000","last_clean":"2023-07-06T01:20:25.834373+0000","last_became_active":"2023-07-06T01:20:24.839201+0000","last_became_peered":"2023-07-06T01:20:24.839201+0000","last_unstale":"2023-07-06T01:20:25.834373+0000","last_undegraded":"2023-07-06T01:20:25.834373+0000","last_fullsized":"2023-07-06T01:20:25.834373+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:57:20.290658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886526+0000","last_change":"2023-07-06T01:20:23.853509+0000","last_active":"2023-07-06T01:20:25.886526+0000","last_peered":"2023-07-06T01:20:25.886526+0000","last_clean":"2023-07-06T01:20:25.886526+0000","last_became_active":"2023-07-06T01:20:23.853060+0000","last_became_peered":"2023-07-06T01:20:23.853060+0000","last_unstale":"2023-07-06T01:20:25.886526+0000","last_undegraded":"2023-07-06T01:20:25.886526+0000","last_fullsized":"2023-07-06T01:20:25.886526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:40.214108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886380+0000","last_change":"2023-07-06T01:20:24.856240+0000","last_active":"2023-07-06T01:20:25.886380+0000","last_peered":"2023-07-06T01:20:25.886380+0000","last_clean":"2023-07-06T01:20:25.886380+0000","last_became_active":"2023-07-06T01:20:24.853426+0000","last_became_peered":"2023-07-06T01:20:24.853426+0000","last_unstale":"2023-07-06T01:20:25.886380+0000","last_undegraded":"2023-07-06T01:20:25.886380+0000","last_fullsized":"2023-07-06T01:20:25.886380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:59.584568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.829932+0000","last_change":"2023-07-06T01:20:22.829932+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.829932+0000","last_undegraded":"2023-07-06T01:20:22.829932+0000","last_fullsized":"2023-07-06T01:20:22.829932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836572+0000","last_change":"2023-07-06T01:20:23.849040+0000","last_active":"2023-07-06T01:20:24.836572+0000","last_peered":"2023-07-06T01:20:24.836572+0000","last_clean":"2023-07-06T01:20:24.836572+0000","last_became_active":"2023-07-06T01:20:23.848756+0000","last_became_peered":"2023-07-06T01:20:23.848756+0000","last_unstale":"2023-07-06T01:20:24.836572+0000","last_undegraded":"2023-07-06T01:20:24.836572+0000","last_fullsized":"2023-07-06T01:20:24.836572+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:05:00.225120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855846+0000","last_change":"2023-07-06T01:20:24.849751+0000","last_active":"2023-07-06T01:20:24.855846+0000","last_peered":"2023-07-06T01:20:24.855846+0000","last_clean":"2023-07-06T01:20:24.855846+0000","last_became_active":"2023-07-06T01:20:24.849475+0000","last_became_peered":"2023-07-06T01:20:24.849475+0000","last_unstale":"2023-07-06T01:20:24.855846+0000","last_undegraded":"2023-07-06T01:20:24.855846+0000","last_fullsized":"2023-07-06T01:20:24.855846+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.826762+0000","last_change":"2023-07-06T01:20:22.826762+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.826762+0000","last_undegraded":"2023-07-06T01:20:22.826762+0000","last_fullsized":"2023-07-06T01:20:22.826762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884424+0000","last_change":"2023-07-06T01:20:24.860540+0000","last_active":"2023-07-06T01:20:25.884424+0000","last_peered":"2023-07-06T01:20:25.884424+0000","last_clean":"2023-07-06T01:20:25.884424+0000","last_became_active":"2023-07-06T01:20:24.859869+0000","last_became_peered":"2023-07-06T01:20:24.859869+0000","last_unstale":"2023-07-06T01:20:25.884424+0000","last_undegraded":"2023-07-06T01:20:25.884424+0000","last_fullsized":"2023-07-06T01:20:25.884424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:23.281099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.845141+0000","last_change":"2023-07-06T01:20:23.848957+0000","last_active":"2023-07-06T01:20:24.845141+0000","last_peered":"2023-07-06T01:20:24.845141+0000","last_clean":"2023-07-06T01:20:24.845141+0000","last_became_active":"2023-07-06T01:20:23.848646+0000","last_became_peered":"2023-07-06T01:20:23.848646+0000","last_unstale":"2023-07-06T01:20:24.845141+0000","last_undegraded":"2023-07-06T01:20:24.845141+0000","last_fullsized":"2023-07-06T01:20:24.845141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:35:50.683696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886834+0000","last_change":"2023-07-06T01:20:23.847858+0000","last_active":"2023-07-06T01:20:25.886834+0000","last_peered":"2023-07-06T01:20:25.886834+0000","last_clean":"2023-07-06T01:20:25.886834+0000","last_became_active":"2023-07-06T01:20:23.847549+0000","last_became_peered":"2023-07-06T01:20:23.847549+0000","last_unstale":"2023-07-06T01:20:25.886834+0000","last_undegraded":"2023-07-06T01:20:25.886834+0000","last_fullsized":"2023-07-06T01:20:25.886834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:53:56.761538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834565+0000","last_change":"2023-07-06T01:20:24.848275+0000","last_active":"2023-07-06T01:20:25.834565+0000","last_peered":"2023-07-06T01:20:25.834565+0000","last_clean":"2023-07-06T01:20:25.834565+0000","last_became_active":"2023-07-06T01:20:24.847949+0000","last_became_peered":"2023-07-06T01:20:24.847949+0000","last_unstale":"2023-07-06T01:20:25.834565+0000","last_undegraded":"2023-07-06T01:20:25.834565+0000","last_fullsized":"2023-07-06T01:20:25.834565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:24:03.775615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887068+0000","last_change":"2023-07-06T01:20:23.849878+0000","last_active":"2023-07-06T01:20:25.887068+0000","last_peered":"2023-07-06T01:20:25.887068+0000","last_clean":"2023-07-06T01:20:25.887068+0000","last_became_active":"2023-07-06T01:20:23.849573+0000","last_became_peered":"2023-07-06T01:20:23.849573+0000","last_unstale":"2023-07-06T01:20:25.887068+0000","last_undegraded":"2023-07-06T01:20:25.887068+0000","last_fullsized":"2023-07-06T01:20:25.887068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:12:26.209870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.868009+0000","last_change":"2023-07-06T01:20:23.845648+0000","last_active":"2023-07-06T01:20:24.868009+0000","last_peered":"2023-07-06T01:20:24.868009+0000","last_clean":"2023-07-06T01:20:24.868009+0000","last_became_active":"2023-07-06T01:20:23.845340+0000","last_became_peered":"2023-07-06T01:20:23.845340+0000","last_unstale":"2023-07-06T01:20:24.868009+0000","last_undegraded":"2023-07-06T01:20:24.868009+0000","last_fullsized":"2023-07-06T01:20:24.868009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:39:35.605373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834521+0000","last_change":"2023-07-06T01:20:24.858282+0000","last_active":"2023-07-06T01:20:25.834521+0000","last_peered":"2023-07-06T01:20:25.834521+0000","last_clean":"2023-07-06T01:20:25.834521+0000","last_became_active":"2023-07-06T01:20:24.857879+0000","last_became_peered":"2023-07-06T01:20:24.857879+0000","last_unstale":"2023-07-06T01:20:25.834521+0000","last_undegraded":"2023-07-06T01:20:25.834521+0000","last_fullsized":"2023-07-06T01:20:25.834521+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:22:46.902965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855788+0000","last_change":"2023-07-06T01:20:23.841745+0000","last_active":"2023-07-06T01:20:24.855788+0000","last_peered":"2023-07-06T01:20:24.855788+0000","last_clean":"2023-07-06T01:20:24.855788+0000","last_became_active":"2023-07-06T01:20:23.841438+0000","last_became_peered":"2023-07-06T01:20:23.841438+0000","last_unstale":"2023-07-06T01:20:24.855788+0000","last_undegraded":"2023-07-06T01:20:24.855788+0000","last_fullsized":"2023-07-06T01:20:24.855788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:54:11.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855590+0000","last_change":"2023-07-06T01:20:24.854272+0000","last_active":"2023-07-06T01:20:24.855590+0000","last_peered":"2023-07-06T01:20:24.855590+0000","last_clean":"2023-07-06T01:20:24.855590+0000","last_became_active":"2023-07-06T01:20:24.854002+0000","last_became_peered":"2023-07-06T01:20:24.854002+0000","last_unstale":"2023-07-06T01:20:24.855590+0000","last_undegraded":"2023-07-06T01:20:24.855590+0000","last_fullsized":"2023-07-06T01:20:24.855590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:23:07.815642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836795+0000","last_change":"2023-07-06T01:20:23.854212+0000","last_active":"2023-07-06T01:20:24.836795+0000","last_peered":"2023-07-06T01:20:24.836795+0000","last_clean":"2023-07-06T01:20:24.836795+0000","last_became_active":"2023-07-06T01:20:23.853928+0000","last_became_peered":"2023-07-06T01:20:23.853928+0000","last_unstale":"2023-07-06T01:20:24.836795+0000","last_undegraded":"2023-07-06T01:20:24.836795+0000","last_fullsized":"2023-07-06T01:20:24.836795+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:49.726864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835969+0000","last_change":"2023-07-06T01:20:23.847446+0000","last_active":"2023-07-06T01:20:25.835969+0000","last_peered":"2023-07-06T01:20:25.835969+0000","last_clean":"2023-07-06T01:20:25.835969+0000","last_became_active":"2023-07-06T01:20:23.846929+0000","last_became_peered":"2023-07-06T01:20:23.846929+0000","last_unstale":"2023-07-06T01:20:25.835969+0000","last_undegraded":"2023-07-06T01:20:25.835969+0000","last_fullsized":"2023-07-06T01:20:25.835969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:05:00.225120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855866+0000","last_change":"2023-07-06T01:20:24.855191+0000","last_active":"2023-07-06T01:20:24.855866+0000","last_peered":"2023-07-06T01:20:24.855866+0000","last_clean":"2023-07-06T01:20:24.855866+0000","last_became_active":"2023-07-06T01:20:24.854898+0000","last_became_peered":"2023-07-06T01:20:24.854898+0000","last_unstale":"2023-07-06T01:20:24.855866+0000","last_undegraded":"2023-07-06T01:20:24.855866+0000","last_fullsized":"2023-07-06T01:20:24.855866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.419832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.829961+0000","last_change":"2023-07-06T01:20:22.829961+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.829961+0000","last_undegraded":"2023-07-06T01:20:22.829961+0000","last_fullsized":"2023-07-06T01:20:22.829961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862166+0000","last_change":"2023-07-06T01:20:24.861072+0000","last_active":"2023-07-06T01:20:24.862166+0000","last_peered":"2023-07-06T01:20:24.862166+0000","last_clean":"2023-07-06T01:20:24.862166+0000","last_became_active":"2023-07-06T01:20:24.860745+0000","last_became_peered":"2023-07-06T01:20:24.860745+0000","last_unstale":"2023-07-06T01:20:24.862166+0000","last_undegraded":"2023-07-06T01:20:24.862166+0000","last_fullsized":"2023-07-06T01:20:24.862166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:45.220808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887003+0000","last_change":"2023-07-06T01:20:23.848801+0000","last_active":"2023-07-06T01:20:25.887003+0000","last_peered":"2023-07-06T01:20:25.887003+0000","last_clean":"2023-07-06T01:20:25.887003+0000","last_became_active":"2023-07-06T01:20:23.848412+0000","last_became_peered":"2023-07-06T01:20:23.848412+0000","last_unstale":"2023-07-06T01:20:25.887003+0000","last_undegraded":"2023-07-06T01:20:25.887003+0000","last_fullsized":"2023-07-06T01:20:25.887003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:35:50.683696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886952+0000","last_change":"2023-07-06T01:20:24.854853+0000","last_active":"2023-07-06T01:20:25.886952+0000","last_peered":"2023-07-06T01:20:25.886952+0000","last_clean":"2023-07-06T01:20:25.886952+0000","last_became_active":"2023-07-06T01:20:24.854159+0000","last_became_peered":"2023-07-06T01:20:24.854159+0000","last_unstale":"2023-07-06T01:20:25.886952+0000","last_undegraded":"2023-07-06T01:20:25.886952+0000","last_fullsized":"2023-07-06T01:20:25.886952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.419832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.826760+0000","last_change":"2023-07-06T01:20:22.826760+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.826760+0000","last_undegraded":"2023-07-06T01:20:22.826760+0000","last_fullsized":"2023-07-06T01:20:22.826760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884278+0000","last_change":"2023-07-06T01:20:23.846883+0000","last_active":"2023-07-06T01:20:25.884278+0000","last_peered":"2023-07-06T01:20:25.884278+0000","last_clean":"2023-07-06T01:20:25.884278+0000","last_became_active":"2023-07-06T01:20:23.846566+0000","last_became_peered":"2023-07-06T01:20:23.846566+0000","last_unstale":"2023-07-06T01:20:25.884278+0000","last_undegraded":"2023-07-06T01:20:25.884278+0000","last_fullsized":"2023-07-06T01:20:25.884278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:44:49.870273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884313+0000","last_change":"2023-07-06T01:20:24.853783+0000","last_active":"2023-07-06T01:20:25.884313+0000","last_peered":"2023-07-06T01:20:25.884313+0000","last_clean":"2023-07-06T01:20:25.884313+0000","last_became_active":"2023-07-06T01:20:24.853427+0000","last_became_peered":"2023-07-06T01:20:24.853427+0000","last_unstale":"2023-07-06T01:20:25.884313+0000","last_undegraded":"2023-07-06T01:20:25.884313+0000","last_fullsized":"2023-07-06T01:20:25.884313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:33:25.311596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.837321+0000","last_change":"2023-07-06T01:20:23.847016+0000","last_active":"2023-07-06T01:20:24.837321+0000","last_peered":"2023-07-06T01:20:24.837321+0000","last_clean":"2023-07-06T01:20:24.837321+0000","last_became_active":"2023-07-06T01:20:23.846748+0000","last_became_peered":"2023-07-06T01:20:23.846748+0000","last_unstale":"2023-07-06T01:20:24.837321+0000","last_undegraded":"2023-07-06T01:20:24.837321+0000","last_fullsized":"2023-07-06T01:20:24.837321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:13.424430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884268+0000","last_change":"2023-07-06T01:20:23.847882+0000","last_active":"2023-07-06T01:20:25.884268+0000","last_peered":"2023-07-06T01:20:25.884268+0000","last_clean":"2023-07-06T01:20:25.884268+0000","last_became_active":"2023-07-06T01:20:23.847534+0000","last_became_peered":"2023-07-06T01:20:23.847534+0000","last_unstale":"2023-07-06T01:20:25.884268+0000","last_undegraded":"2023-07-06T01:20:25.884268+0000","last_fullsized":"2023-07-06T01:20:25.884268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:22:12.321678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834687+0000","last_change":"2023-07-06T01:20:24.848033+0000","last_active":"2023-07-06T01:20:25.834687+0000","last_peered":"2023-07-06T01:20:25.834687+0000","last_clean":"2023-07-06T01:20:25.834687+0000","last_became_active":"2023-07-06T01:20:24.847514+0000","last_became_peered":"2023-07-06T01:20:24.847514+0000","last_unstale":"2023-07-06T01:20:25.834687+0000","last_undegraded":"2023-07-06T01:20:25.834687+0000","last_fullsized":"2023-07-06T01:20:25.834687+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:02.805903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884363+0000","last_change":"2023-07-06T01:20:23.850960+0000","last_active":"2023-07-06T01:20:25.884363+0000","last_peered":"2023-07-06T01:20:25.884363+0000","last_clean":"2023-07-06T01:20:25.884363+0000","last_became_active":"2023-07-06T01:20:23.850699+0000","last_became_peered":"2023-07-06T01:20:23.850699+0000","last_unstale":"2023-07-06T01:20:25.884363+0000","last_undegraded":"2023-07-06T01:20:25.884363+0000","last_fullsized":"2023-07-06T01:20:25.884363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:30:39.863076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886519+0000","last_change":"2023-07-06T01:20:24.861071+0000","last_active":"2023-07-06T01:20:25.886519+0000","last_peered":"2023-07-06T01:20:25.886519+0000","last_clean":"2023-07-06T01:20:25.886519+0000","last_became_active":"2023-07-06T01:20:24.860803+0000","last_became_peered":"2023-07-06T01:20:24.860803+0000","last_unstale":"2023-07-06T01:20:25.886519+0000","last_undegraded":"2023-07-06T01:20:25.886519+0000","last_fullsized":"2023-07-06T01:20:25.886519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:17.181181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.830814+0000","last_change":"2023-07-06T01:20:22.830814+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.830814+0000","last_undegraded":"2023-07-06T01:20:22.830814+0000","last_fullsized":"2023-07-06T01:20:22.830814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862183+0000","last_change":"2023-07-06T01:20:24.861226+0000","last_active":"2023-07-06T01:20:24.862183+0000","last_peered":"2023-07-06T01:20:24.862183+0000","last_clean":"2023-07-06T01:20:24.862183+0000","last_became_active":"2023-07-06T01:20:24.860944+0000","last_became_peered":"2023-07-06T01:20:24.860944+0000","last_unstale":"2023-07-06T01:20:24.862183+0000","last_undegraded":"2023-07-06T01:20:24.862183+0000","last_fullsized":"2023-07-06T01:20:24.862183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:12.103457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884125+0000","last_change":"2023-07-06T01:20:23.849860+0000","last_active":"2023-07-06T01:20:25.884125+0000","last_peered":"2023-07-06T01:20:25.884125+0000","last_clean":"2023-07-06T01:20:25.884125+0000","last_became_active":"2023-07-06T01:20:23.849507+0000","last_became_peered":"2023-07-06T01:20:23.849507+0000","last_unstale":"2023-07-06T01:20:25.884125+0000","last_undegraded":"2023-07-06T01:20:25.884125+0000","last_fullsized":"2023-07-06T01:20:25.884125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:51.413216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.826768+0000","last_change":"2023-07-06T01:20:22.826768+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.826768+0000","last_undegraded":"2023-07-06T01:20:22.826768+0000","last_fullsized":"2023-07-06T01:20:22.826768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T01:20:22.827804+0000","last_change":"2023-07-06T01:19:20.074195+0000","last_active":"2023-07-06T01:20:22.827804+0000","last_peered":"2023-07-06T01:20:22.827804+0000","last_clean":"2023-07-06T01:20:22.827804+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:20:22.827804+0000","last_undegraded":"2023-07-06T01:20:22.827804+0000","last_fullsized":"2023-07-06T01:20:22.827804+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:38:31.352462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.842093+0000","last_change":"2023-07-06T01:20:23.849570+0000","last_active":"2023-07-06T01:20:24.842093+0000","last_peered":"2023-07-06T01:20:24.842093+0000","last_clean":"2023-07-06T01:20:24.842093+0000","last_became_active":"2023-07-06T01:20:23.849255+0000","last_became_peered":"2023-07-06T01:20:23.849255+0000","last_unstale":"2023-07-06T01:20:24.842093+0000","last_undegraded":"2023-07-06T01:20:24.842093+0000","last_fullsized":"2023-07-06T01:20:24.842093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:53:56.761538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.846124+0000","last_change":"2023-07-06T01:19:19.587110+0000","last_active":"2023-07-06T01:20:24.846124+0000","last_peered":"2023-07-06T01:20:24.846124+0000","last_clean":"2023-07-06T01:20:24.846124+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:20:24.846124+0000","last_undegraded":"2023-07-06T01:20:24.846124+0000","last_fullsized":"2023-07-06T01:20:24.846124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:49:50.820241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859472+0000","last_change":"2023-07-06T01:20:24.858252+0000","last_active":"2023-07-06T01:20:24.859472+0000","last_peered":"2023-07-06T01:20:24.859472+0000","last_clean":"2023-07-06T01:20:24.859472+0000","last_became_active":"2023-07-06T01:20:24.856691+0000","last_became_peered":"2023-07-06T01:20:24.856691+0000","last_unstale":"2023-07-06T01:20:24.859472+0000","last_undegraded":"2023-07-06T01:20:24.859472+0000","last_fullsized":"2023-07-06T01:20:24.859472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:03.093141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887051+0000","last_change":"2023-07-06T01:20:23.847756+0000","last_active":"2023-07-06T01:20:25.887051+0000","last_peered":"2023-07-06T01:20:25.887051+0000","last_clean":"2023-07-06T01:20:25.887051+0000","last_became_active":"2023-07-06T01:20:23.847392+0000","last_became_peered":"2023-07-06T01:20:23.847392+0000","last_unstale":"2023-07-06T01:20:25.887051+0000","last_undegraded":"2023-07-06T01:20:25.887051+0000","last_fullsized":"2023-07-06T01:20:25.887051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:50:55.073152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":35,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836206+0000","last_change":"2023-07-06T01:19:19.584242+0000","last_active":"2023-07-06T01:20:24.836206+0000","last_peered":"2023-07-06T01:20:24.836206+0000","last_clean":"2023-07-06T01:20:24.836206+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:20:24.836206+0000","last_undegraded":"2023-07-06T01:20:24.836206+0000","last_fullsized":"2023-07-06T01:20:24.836206+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:02:18.883377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":183,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T01:20:22.831308+0000","last_change":"2023-07-06T01:19:19.586495+0000","last_active":"2023-07-06T01:20:22.831308+0000","last_peered":"2023-07-06T01:20:22.831308+0000","last_clean":"2023-07-06T01:20:22.831308+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:20:22.831308+0000","last_undegraded":"2023-07-06T01:20:22.831308+0000","last_fullsized":"2023-07-06T01:20:22.831308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:06:26.958597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.885927+0000","last_change":"2023-07-06T01:20:23.854385+0000","last_active":"2023-07-06T01:20:25.885927+0000","last_peered":"2023-07-06T01:20:25.885927+0000","last_clean":"2023-07-06T01:20:25.885927+0000","last_became_active":"2023-07-06T01:20:23.853982+0000","last_became_peered":"2023-07-06T01:20:23.853982+0000","last_unstale":"2023-07-06T01:20:25.885927+0000","last_undegraded":"2023-07-06T01:20:25.885927+0000","last_fullsized":"2023-07-06T01:20:25.885927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:01.772125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T01:20:22.831236+0000","last_change":"2023-07-06T01:19:19.586426+0000","last_active":"2023-07-06T01:20:22.831236+0000","last_peered":"2023-07-06T01:20:22.831236+0000","last_clean":"2023-07-06T01:20:22.831236+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:20:22.831236+0000","last_undegraded":"2023-07-06T01:20:22.831236+0000","last_fullsized":"2023-07-06T01:20:22.831236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:46.933540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886966+0000","last_change":"2023-07-06T01:20:24.855685+0000","last_active":"2023-07-06T01:20:25.886966+0000","last_peered":"2023-07-06T01:20:25.886966+0000","last_clean":"2023-07-06T01:20:25.886966+0000","last_became_active":"2023-07-06T01:20:24.855347+0000","last_became_peered":"2023-07-06T01:20:24.855347+0000","last_unstale":"2023-07-06T01:20:25.886966+0000","last_undegraded":"2023-07-06T01:20:25.886966+0000","last_fullsized":"2023-07-06T01:20:25.886966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:39:36.610034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.830379+0000","last_change":"2023-07-06T01:20:22.830379+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.830379+0000","last_undegraded":"2023-07-06T01:20:22.830379+0000","last_fullsized":"2023-07-06T01:20:22.830379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T01:20:22.831212+0000","last_change":"2023-07-06T01:19:19.586626+0000","last_active":"2023-07-06T01:20:22.831212+0000","last_peered":"2023-07-06T01:20:22.831212+0000","last_clean":"2023-07-06T01:20:22.831212+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:20:22.831212+0000","last_undegraded":"2023-07-06T01:20:22.831212+0000","last_fullsized":"2023-07-06T01:20:22.831212+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:06:51.308527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.852493+0000","last_change":"2023-07-06T01:20:24.845956+0000","last_active":"2023-07-06T01:20:24.852493+0000","last_peered":"2023-07-06T01:20:24.852493+0000","last_clean":"2023-07-06T01:20:24.852493+0000","last_became_active":"2023-07-06T01:20:24.845698+0000","last_became_peered":"2023-07-06T01:20:24.845698+0000","last_unstale":"2023-07-06T01:20:24.852493+0000","last_undegraded":"2023-07-06T01:20:24.852493+0000","last_fullsized":"2023-07-06T01:20:24.852493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:41.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834898+0000","last_change":"2023-07-06T01:20:23.847902+0000","last_active":"2023-07-06T01:20:25.834898+0000","last_peered":"2023-07-06T01:20:25.834898+0000","last_clean":"2023-07-06T01:20:25.834898+0000","last_became_active":"2023-07-06T01:20:23.847543+0000","last_became_peered":"2023-07-06T01:20:23.847543+0000","last_unstale":"2023-07-06T01:20:25.834898+0000","last_undegraded":"2023-07-06T01:20:25.834898+0000","last_fullsized":"2023-07-06T01:20:25.834898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:51.462337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886549+0000","last_change":"2023-07-06T01:19:19.584086+0000","last_active":"2023-07-06T01:20:25.886549+0000","last_peered":"2023-07-06T01:20:25.886549+0000","last_clean":"2023-07-06T01:20:25.886549+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:20:25.886549+0000","last_undegraded":"2023-07-06T01:20:25.886549+0000","last_fullsized":"2023-07-06T01:20:25.886549+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:01:33.579869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855022+0000","last_change":"2023-07-06T01:20:24.851662+0000","last_active":"2023-07-06T01:20:24.855022+0000","last_peered":"2023-07-06T01:20:24.855022+0000","last_clean":"2023-07-06T01:20:24.855022+0000","last_became_active":"2023-07-06T01:20:24.851349+0000","last_became_peered":"2023-07-06T01:20:24.851349+0000","last_unstale":"2023-07-06T01:20:24.855022+0000","last_undegraded":"2023-07-06T01:20:24.855022+0000","last_fullsized":"2023-07-06T01:20:24.855022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:50.421447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836073+0000","last_change":"2023-07-06T01:20:23.845197+0000","last_active":"2023-07-06T01:20:25.836073+0000","last_peered":"2023-07-06T01:20:25.836073+0000","last_clean":"2023-07-06T01:20:25.836073+0000","last_became_active":"2023-07-06T01:20:23.844508+0000","last_became_peered":"2023-07-06T01:20:23.844508+0000","last_unstale":"2023-07-06T01:20:25.836073+0000","last_undegraded":"2023-07-06T01:20:25.836073+0000","last_fullsized":"2023-07-06T01:20:25.836073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:00.176807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836633+0000","last_change":"2023-07-06T01:19:19.586424+0000","last_active":"2023-07-06T01:20:25.836633+0000","last_peered":"2023-07-06T01:20:25.836633+0000","last_clean":"2023-07-06T01:20:25.836633+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:20:25.836633+0000","last_undegraded":"2023-07-06T01:20:25.836633+0000","last_fullsized":"2023-07-06T01:20:25.836633+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:04:57.410877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834442+0000","last_change":"2023-07-06T01:20:24.849984+0000","last_active":"2023-07-06T01:20:25.834442+0000","last_peered":"2023-07-06T01:20:25.834442+0000","last_clean":"2023-07-06T01:20:25.834442+0000","last_became_active":"2023-07-06T01:20:24.849603+0000","last_became_peered":"2023-07-06T01:20:24.849603+0000","last_unstale":"2023-07-06T01:20:25.834442+0000","last_undegraded":"2023-07-06T01:20:25.834442+0000","last_fullsized":"2023-07-06T01:20:25.834442+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:39:50.052979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834826+0000","last_change":"2023-07-06T01:20:23.842452+0000","last_active":"2023-07-06T01:20:25.834826+0000","last_peered":"2023-07-06T01:20:25.834826+0000","last_clean":"2023-07-06T01:20:25.834826+0000","last_became_active":"2023-07-06T01:20:23.842176+0000","last_became_peered":"2023-07-06T01:20:23.842176+0000","last_unstale":"2023-07-06T01:20:25.834826+0000","last_undegraded":"2023-07-06T01:20:25.834826+0000","last_fullsized":"2023-07-06T01:20:25.834826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:16.176519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836914+0000","last_change":"2023-07-06T01:19:19.583699+0000","last_active":"2023-07-06T01:20:25.836914+0000","last_peered":"2023-07-06T01:20:25.836914+0000","last_clean":"2023-07-06T01:20:25.836914+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:20:25.836914+0000","last_undegraded":"2023-07-06T01:20:25.836914+0000","last_fullsized":"2023-07-06T01:20:25.836914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:52:09.171519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":94,"acting":94,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":721739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.505},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.439},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45200000000000001}]},{"osd":3,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53200000000000003}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.34100000000000003}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.375}]}]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":344064,"data_stored":137541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.499},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47599999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.436},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.39800000000000002}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.622}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":966656,"data_stored":741755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":12.23,"5min":12.23,"15min":12.23},"last":0.311},{"interface":"front","average":{"1min":1.839,"5min":1.839,"15min":1.839},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":15.208,"5min":15.208,"15min":15.208},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":12.074,"5min":12.074,"15min":12.074},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":11.988,"5min":11.988,"15min":11.988},"last":0.73999999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.173,"5min":2.173,"15min":2.173},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"last":0.442},{"interface":"front","average":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":15.244999999999999,"5min":15.244999999999999,"15min":15.244999999999999},"last":0.379}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":11.923,"5min":11.923,"15min":11.923},"last":2.6080000000000001},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":12.019,"5min":12.019,"15min":12.019},"last":3.5950000000000002}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":11.999000000000001,"5min":11.999000000000001,"15min":11.999000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":11.951000000000001,"5min":11.951000000000001,"15min":11.951000000000001},"last":0.624}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":15.167999999999999,"5min":15.167999999999999,"15min":15.167999999999999},"last":2.7490000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":12.041,"5min":12.041,"15min":12.041},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":15.115,"5min":15.115,"15min":15.115},"last":1.625},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":11.965,"5min":11.965,"15min":11.965},"last":1.8200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":157984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":0.251}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.215},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.27200000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.503},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":6.3049999999999997,"5min":6.3049999999999997,"15min":6.3049999999999997},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.2430000000000003,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.63900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.436},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.625}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54900000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":152895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.307},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.375}]},{"osd":2,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.32700000000000001}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":3.8130000000000002,"5min":3.8130000000000002,"15min":3.8130000000000002},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.71899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":158062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.5},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.436}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":158532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.629},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.505},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.439}]},{"osd":5,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42399999999999999}]},{"osd":7,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:20:29.399 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:20:29.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.558+0000 7f6c38709700 1 Processor -- start 2023-07-06T01:20:29.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.558+0000 7f6c38709700 1 -- start start 2023-07-06T01:20:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.559+0000 7f6c38709700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f6c30158bb0 0x7f6c3006b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.559+0000 7f6c38709700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c30158f80 con 0x7f6c3015c6b0 2023-07-06T01:20:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.559+0000 7f6c38709700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3015ce90 con 0x7f6c3006c900 2023-07-06T01:20:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.559+0000 7f6c38709700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c300701e0 con 0x7f6c30158bb0 2023-07-06T01:20:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.559+0000 7f6c35ca4700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6c3015c6b0 0x7f6c3006c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38194/0 (socket says 172.21.15.22:38194) 2023-07-06T01:20:29.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.559+0000 7f6c35ca4700 1 -- 172.21.15.22:0/1502152433 learned_addr learned my addr 172.21.15.22:0/1502152433 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:29.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491348146 0 0) 0x7f6c30158f80 con 0x7f6c3015c6b0 2023-07-06T01:20:29.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c364a5700 1 --2- 172.21.15.22:0/1502152433 >> v2:172.21.15.103:3300/0 conn(0x7f6c30158bb0 0x7f6c3006b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c1c0035b0 con 0x7f6c3015c6b0 2023-07-06T01:20:29.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2557556519 0 0) 0x7f6c1c0035b0 con 0x7f6c3015c6b0 2023-07-06T01:20:29.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 >> v2:172.21.15.103:3300/0 conn(0x7f6c30158bb0 msgr2=0x7f6c3006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:29.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 --2- 172.21.15.22:0/1502152433 >> v2:172.21.15.103:3300/0 conn(0x7f6c30158bb0 0x7f6c3006b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 >> v1:172.21.15.103:6789/0 conn(0x7f6c3006c900 legacy=0x7f6c30070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3015d170 con 0x7f6c3015c6b0 2023-07-06T01:20:29.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f6c2c001670 con 0x7f6c3015c6b0 2023-07-06T01:20:29.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 >> v1:172.21.15.22:6789/0 conn(0x7f6c3015c6b0 legacy=0x7f6c3006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 --2- 172.21.15.22:0/1502152433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c1c003a00 0x7f6c1c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 --2- 172.21.15.22:0/1502152433 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c1c004850 0x7f6c1c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 --2- 172.21.15.22:0/1502152433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c005200 0x7f6c1c00cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.565 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300701e0 con 0x7f6c1c003a00 2023-07-06T01:20:29.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c00d190 con 0x7f6c1c005200 2023-07-06T01:20:29.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c00d350 con 0x7f6c1c004850 2023-07-06T01:20:29.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c364a5700 1 --2- 172.21.15.22:0/1502152433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c1c003a00 0x7f6c1c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c36ca6700 1 --2- 172.21.15.22:0/1502152433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c005200 0x7f6c1c00cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c35ca4700 1 --2- 172.21.15.22:0/1502152433 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c1c004850 0x7f6c1c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c364a5700 1 -- 172.21.15.22:0/1502152433 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c1c004850 msgr2=0x7f6c1c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:29.566 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c364a5700 1 --2- 172.21.15.22:0/1502152433 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c1c004850 0x7f6c1c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c364a5700 1 -- 172.21.15.22:0/1502152433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c005200 msgr2=0x7f6c1c00cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c364a5700 1 --2- 172.21.15.22:0/1502152433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c005200 0x7f6c1c00cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c364a5700 1 -- 172.21.15.22:0/1502152433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c0035b0 con 0x7f6c1c003a00 2023-07-06T01:20:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c35ca4700 1 --2- 172.21.15.22:0/1502152433 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c1c004850 0x7f6c1c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c36ca6700 1 --2- 172.21.15.22:0/1502152433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c1c005200 0x7f6c1c00cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.560+0000 7f6c36ca6700 1 -- 172.21.15.22:0/1502152433 reap_dead start 2023-07-06T01:20:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.561+0000 7f6c364a5700 1 --2- 172.21.15.22:0/1502152433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c1c003a00 0x7f6c1c003e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009b60 tx=0x7f6c1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521cb9206f0a7fb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:29.567 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.561+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000e6c0 con 0x7f6c1c003a00 2023-07-06T01:20:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.561+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2000e820 con 0x7f6c1c003a00 2023-07-06T01:20:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.561+0000 7f6c354a3700 1 -- 172.21.15.22:0/1502152433 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000eb90 con 0x7f6c1c003a00 2023-07-06T01:20:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.562+0000 7f6c38709700 1 -- 172.21.15.22:0/1502152433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c1c003a00 msgr2=0x7f6c1c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.562+0000 7f6c38709700 1 --2- 172.21.15.22:0/1502152433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c1c003a00 0x7f6c1c003e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009b60 tx=0x7f6c1c005880 comp rx=0 tx=0).stop 2023-07-06T01:20:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.562+0000 7f6c38709700 1 -- 172.21.15.22:0/1502152433 shutdown_connections 2023-07-06T01:20:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.562+0000 7f6c38709700 1 --2- 172.21.15.22:0/1502152433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c1c003a00 0x7f6c1c003e10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.562+0000 7f6c38709700 1 -- 172.21.15.22:0/1502152433 >> 172.21.15.22:0/1502152433 conn(0x7f6c300ccb00 msgr2=0x7f6c300cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:29.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.562+0000 7f6c38709700 1 -- 172.21.15.22:0/1502152433 shutdown_connections 2023-07-06T01:20:29.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.562+0000 7f6c38709700 1 -- 172.21.15.22:0/1502152433 wait complete. 2023-07-06T01:20:29.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.563+0000 7f6c38709700 1 Processor -- start 2023-07-06T01:20:29.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.563+0000 7f6c38709700 1 -- start start 2023-07-06T01:20:29.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.564+0000 7f6c38709700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c1c003a00 0x7f6c3014ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.564+0000 7f6c38709700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c300c4870 0x7f6c300b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.564+0000 7f6c38709700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c300adfa0 0x7f6c300af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.564+0000 7f6c38709700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300ae760 con 0x7f6c1c003a00 2023-07-06T01:20:29.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.564+0000 7f6c38709700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300c3ab0 con 0x7f6c300adfa0 2023-07-06T01:20:29.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.564+0000 7f6c38709700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300c3d90 con 0x7f6c300c4870 2023-07-06T01:20:29.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.564+0000 7f6c364a5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c1c003a00 0x7f6c3014ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c364a5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c1c003a00 0x7f6c3014ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47460/0 (socket says 172.21.15.22:47460) 2023-07-06T01:20:29.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c364a5700 1 -- 172.21.15.22:0/3986163662 learned_addr learned my addr 172.21.15.22:0/3986163662 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:29.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c35ca4700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c300c4870 0x7f6c300b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c36ca6700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c300adfa0 0x7f6c300af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c364a5700 1 -- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c300c4870 msgr2=0x7f6c300b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c364a5700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c300c4870 0x7f6c300b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c364a5700 1 -- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c300adfa0 msgr2=0x7f6c300af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c364a5700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c300adfa0 0x7f6c300af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c364a5700 1 -- 172.21.15.22:0/3986163662 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c300c4070 con 0x7f6c1c003a00 2023-07-06T01:20:29.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c364a5700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c1c003a00 0x7f6c3014ca80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00d160 tx=0x7f6c200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:29.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c26ffd700 1 -- 172.21.15.22:0/3986163662 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000e6c0 con 0x7f6c1c003a00 2023-07-06T01:20:29.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c26ffd700 1 -- 172.21.15.22:0/3986163662 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c20003ad0 con 0x7f6c1c003a00 2023-07-06T01:20:29.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c38709700 1 -- 172.21.15.22:0/3986163662 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c300c4350 con 0x7f6c1c003a00 2023-07-06T01:20:29.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c26ffd700 1 -- 172.21.15.22:0/3986163662 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20003e80 con 0x7f6c1c003a00 2023-07-06T01:20:29.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.565+0000 7f6c38709700 1 -- 172.21.15.22:0/3986163662 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c300aff10 con 0x7f6c1c003a00 2023-07-06T01:20:29.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.566+0000 7f6c38709700 1 -- 172.21.15.22:0/3986163662 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c300c46c0 con 0x7f6c1c003a00 2023-07-06T01:20:29.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.567+0000 7f6c26ffd700 1 -- 172.21.15.22:0/3986163662 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c20013470 con 0x7f6c1c003a00 2023-07-06T01:20:29.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.567+0000 7f6c26ffd700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6c1406ace0 0x7f6c1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.567+0000 7f6c26ffd700 1 -- 172.21.15.22:0/3986163662 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c2008ac30 con 0x7f6c1c003a00 2023-07-06T01:20:29.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.567+0000 7f6c35ca4700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6c1406ace0 0x7f6c1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.568+0000 7f6c35ca4700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6c1406ace0 0x7f6c1406d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c300b0620 tx=0x7f6c2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:29.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.572+0000 7f6c26ffd700 1 -- 172.21.15.22:0/3986163662 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6c20055e90 con 0x7f6c1c003a00 2023-07-06T01:20:29.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.765+0000 7f6c38709700 1 -- 172.21.15.22:0/3986163662 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c300c38a0 con 0x7f6c1406ace0 2023-07-06T01:20:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.774+0000 7f6c26ffd700 1 -- 172.21.15.22:0/3986163662 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308073 (secure 0 0 0) 0x7f6c300c38a0 con 0x7f6c1406ace0 2023-07-06T01:20:29.779 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.778+0000 7f6c38709700 1 -- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6c1406ace0 msgr2=0x7f6c1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.778+0000 7f6c38709700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6c1406ace0 0x7f6c1406d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c300b0620 tx=0x7f6c2c009040 comp rx=0 tx=0).stop 2023-07-06T01:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.778+0000 7f6c38709700 1 -- 172.21.15.22:0/3986163662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c1c003a00 msgr2=0x7f6c3014ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.778+0000 7f6c38709700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c1c003a00 0x7f6c3014ca80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00d160 tx=0x7f6c200047c0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.778+0000 7f6c38709700 1 -- 172.21.15.22:0/3986163662 shutdown_connections 2023-07-06T01:20:29.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.778+0000 7f6c38709700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6c1406ace0 0x7f6c1406d190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.778+0000 7f6c38709700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c300adfa0 0x7f6c300af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.778+0000 7f6c38709700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c1c003a00 0x7f6c3014ca80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.778+0000 7f6c38709700 1 --2- 172.21.15.22:0/3986163662 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6c300c4870 0x7f6c300b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.778+0000 7f6c38709700 1 -- 172.21.15.22:0/3986163662 >> 172.21.15.22:0/3986163662 conn(0x7f6c300ccb00 msgr2=0x7f6c30070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:29.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.778+0000 7f6c38709700 1 -- 172.21.15.22:0/3986163662 shutdown_connections 2023-07-06T01:20:29.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.778+0000 7f6c38709700 1 -- 172.21.15.22:0/3986163662 wait complete. 2023-07-06T01:20:29.783 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:20:29.792 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-07-06T01:20:28.014418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":186,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219684,"kb_used_data":4068,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755036,"statfs":{"total":773094113280,"available":772869156864,"internally_reserved":0,"allocated":4165632,"data_stored":2386528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.189545"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855626+0000","last_change":"2023-07-06T01:20:24.852545+0000","last_active":"2023-07-06T01:20:24.855626+0000","last_peered":"2023-07-06T01:20:24.855626+0000","last_clean":"2023-07-06T01:20:24.855626+0000","last_became_active":"2023-07-06T01:20:24.852249+0000","last_became_peered":"2023-07-06T01:20:24.852249+0000","last_unstale":"2023-07-06T01:20:24.855626+0000","last_undegraded":"2023-07-06T01:20:24.855626+0000","last_fullsized":"2023-07-06T01:20:24.855626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:17.181181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.854187+0000","last_change":"2023-07-06T01:20:24.850725+0000","last_active":"2023-07-06T01:20:24.854187+0000","last_peered":"2023-07-06T01:20:24.854187+0000","last_clean":"2023-07-06T01:20:24.854187+0000","last_became_active":"2023-07-06T01:20:24.849359+0000","last_became_peered":"2023-07-06T01:20:24.849359+0000","last_unstale":"2023-07-06T01:20:24.854187+0000","last_undegraded":"2023-07-06T01:20:24.854187+0000","last_fullsized":"2023-07-06T01:20:24.854187+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:20.409326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834789+0000","last_change":"2023-07-06T01:20:24.855847+0000","last_active":"2023-07-06T01:20:25.834789+0000","last_peered":"2023-07-06T01:20:25.834789+0000","last_clean":"2023-07-06T01:20:25.834789+0000","last_became_active":"2023-07-06T01:20:24.855571+0000","last_became_peered":"2023-07-06T01:20:24.855571+0000","last_unstale":"2023-07-06T01:20:25.834789+0000","last_undegraded":"2023-07-06T01:20:25.834789+0000","last_fullsized":"2023-07-06T01:20:25.834789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884281+0000","last_change":"2023-07-06T01:20:24.860708+0000","last_active":"2023-07-06T01:20:25.884281+0000","last_peered":"2023-07-06T01:20:25.884281+0000","last_clean":"2023-07-06T01:20:25.884281+0000","last_became_active":"2023-07-06T01:20:24.860220+0000","last_became_peered":"2023-07-06T01:20:24.860220+0000","last_unstale":"2023-07-06T01:20:25.884281+0000","last_undegraded":"2023-07-06T01:20:25.884281+0000","last_fullsized":"2023-07-06T01:20:25.884281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:42:33.239141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884127+0000","last_change":"2023-07-06T01:20:24.850342+0000","last_active":"2023-07-06T01:20:25.884127+0000","last_peered":"2023-07-06T01:20:25.884127+0000","last_clean":"2023-07-06T01:20:25.884127+0000","last_became_active":"2023-07-06T01:20:24.850083+0000","last_became_peered":"2023-07-06T01:20:24.850083+0000","last_unstale":"2023-07-06T01:20:25.884127+0000","last_undegraded":"2023-07-06T01:20:25.884127+0000","last_fullsized":"2023-07-06T01:20:25.884127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:46:35.434349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.850816+0000","last_change":"2023-07-06T01:20:24.848000+0000","last_active":"2023-07-06T01:20:24.850816+0000","last_peered":"2023-07-06T01:20:24.850816+0000","last_clean":"2023-07-06T01:20:24.850816+0000","last_became_active":"2023-07-06T01:20:24.847814+0000","last_became_peered":"2023-07-06T01:20:24.847814+0000","last_unstale":"2023-07-06T01:20:24.850816+0000","last_undegraded":"2023-07-06T01:20:24.850816+0000","last_fullsized":"2023-07-06T01:20:24.850816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:51.766923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886546+0000","last_change":"2023-07-06T01:20:24.856730+0000","last_active":"2023-07-06T01:20:25.886546+0000","last_peered":"2023-07-06T01:20:25.886546+0000","last_clean":"2023-07-06T01:20:25.886546+0000","last_became_active":"2023-07-06T01:20:24.856460+0000","last_became_peered":"2023-07-06T01:20:24.856460+0000","last_unstale":"2023-07-06T01:20:25.886546+0000","last_undegraded":"2023-07-06T01:20:25.886546+0000","last_fullsized":"2023-07-06T01:20:25.886546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:52.466998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859689+0000","last_change":"2023-07-06T01:20:24.858297+0000","last_active":"2023-07-06T01:20:24.859689+0000","last_peered":"2023-07-06T01:20:24.859689+0000","last_clean":"2023-07-06T01:20:24.859689+0000","last_became_active":"2023-07-06T01:20:24.858033+0000","last_became_peered":"2023-07-06T01:20:24.858033+0000","last_unstale":"2023-07-06T01:20:24.859689+0000","last_undegraded":"2023-07-06T01:20:24.859689+0000","last_fullsized":"2023-07-06T01:20:24.859689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:38.837442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834859+0000","last_change":"2023-07-06T01:20:24.859281+0000","last_active":"2023-07-06T01:20:25.834859+0000","last_peered":"2023-07-06T01:20:25.834859+0000","last_clean":"2023-07-06T01:20:25.834859+0000","last_became_active":"2023-07-06T01:20:24.858982+0000","last_became_peered":"2023-07-06T01:20:24.858982+0000","last_unstale":"2023-07-06T01:20:25.834859+0000","last_undegraded":"2023-07-06T01:20:25.834859+0000","last_fullsized":"2023-07-06T01:20:25.834859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:41.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886707+0000","last_change":"2023-07-06T01:20:24.854480+0000","last_active":"2023-07-06T01:20:25.886707+0000","last_peered":"2023-07-06T01:20:25.886707+0000","last_clean":"2023-07-06T01:20:25.886707+0000","last_became_active":"2023-07-06T01:20:24.854116+0000","last_became_peered":"2023-07-06T01:20:24.854116+0000","last_unstale":"2023-07-06T01:20:25.886707+0000","last_undegraded":"2023-07-06T01:20:25.886707+0000","last_fullsized":"2023-07-06T01:20:25.886707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:12.103457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886974+0000","last_change":"2023-07-06T01:20:24.854685+0000","last_active":"2023-07-06T01:20:25.886974+0000","last_peered":"2023-07-06T01:20:25.886974+0000","last_clean":"2023-07-06T01:20:25.886974+0000","last_became_active":"2023-07-06T01:20:24.853605+0000","last_became_peered":"2023-07-06T01:20:24.853605+0000","last_unstale":"2023-07-06T01:20:25.886974+0000","last_undegraded":"2023-07-06T01:20:25.886974+0000","last_fullsized":"2023-07-06T01:20:25.886974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:54:12.127760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834625+0000","last_change":"2023-07-06T01:20:24.853728+0000","last_active":"2023-07-06T01:20:25.834625+0000","last_peered":"2023-07-06T01:20:25.834625+0000","last_clean":"2023-07-06T01:20:25.834625+0000","last_became_active":"2023-07-06T01:20:24.853427+0000","last_became_peered":"2023-07-06T01:20:24.853427+0000","last_unstale":"2023-07-06T01:20:25.834625+0000","last_undegraded":"2023-07-06T01:20:25.834625+0000","last_fullsized":"2023-07-06T01:20:25.834625+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:31.557538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886831+0000","last_change":"2023-07-06T01:20:24.856012+0000","last_active":"2023-07-06T01:20:25.886831+0000","last_peered":"2023-07-06T01:20:25.886831+0000","last_clean":"2023-07-06T01:20:25.886831+0000","last_became_active":"2023-07-06T01:20:24.855762+0000","last_became_peered":"2023-07-06T01:20:24.855762+0000","last_unstale":"2023-07-06T01:20:25.886831+0000","last_undegraded":"2023-07-06T01:20:25.886831+0000","last_fullsized":"2023-07-06T01:20:25.886831+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:45.220808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886397+0000","last_change":"2023-07-06T01:20:24.857423+0000","last_active":"2023-07-06T01:20:25.886397+0000","last_peered":"2023-07-06T01:20:25.886397+0000","last_clean":"2023-07-06T01:20:25.886397+0000","last_became_active":"2023-07-06T01:20:24.857164+0000","last_became_peered":"2023-07-06T01:20:24.857164+0000","last_unstale":"2023-07-06T01:20:25.886397+0000","last_undegraded":"2023-07-06T01:20:25.886397+0000","last_fullsized":"2023-07-06T01:20:25.886397+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836700+0000","last_change":"2023-07-06T01:20:24.860237+0000","last_active":"2023-07-06T01:20:25.836700+0000","last_peered":"2023-07-06T01:20:25.836700+0000","last_clean":"2023-07-06T01:20:25.836700+0000","last_became_active":"2023-07-06T01:20:24.859324+0000","last_became_peered":"2023-07-06T01:20:24.859324+0000","last_unstale":"2023-07-06T01:20:25.836700+0000","last_undegraded":"2023-07-06T01:20:25.836700+0000","last_fullsized":"2023-07-06T01:20:25.836700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:50.731526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884196+0000","last_change":"2023-07-06T01:20:24.852638+0000","last_active":"2023-07-06T01:20:25.884196+0000","last_peered":"2023-07-06T01:20:25.884196+0000","last_clean":"2023-07-06T01:20:25.884196+0000","last_became_active":"2023-07-06T01:20:24.852341+0000","last_became_peered":"2023-07-06T01:20:24.852341+0000","last_unstale":"2023-07-06T01:20:25.884196+0000","last_undegraded":"2023-07-06T01:20:25.884196+0000","last_fullsized":"2023-07-06T01:20:25.884196+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:17:34.632929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836872+0000","last_change":"2023-07-06T01:20:24.859626+0000","last_active":"2023-07-06T01:20:25.836872+0000","last_peered":"2023-07-06T01:20:25.836872+0000","last_clean":"2023-07-06T01:20:25.836872+0000","last_became_active":"2023-07-06T01:20:24.859274+0000","last_became_peered":"2023-07-06T01:20:24.859274+0000","last_unstale":"2023-07-06T01:20:25.836872+0000","last_undegraded":"2023-07-06T01:20:25.836872+0000","last_fullsized":"2023-07-06T01:20:25.836872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:03:24.140950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886696+0000","last_change":"2023-07-06T01:20:24.854443+0000","last_active":"2023-07-06T01:20:25.886696+0000","last_peered":"2023-07-06T01:20:25.886696+0000","last_clean":"2023-07-06T01:20:25.886696+0000","last_became_active":"2023-07-06T01:20:24.853256+0000","last_became_peered":"2023-07-06T01:20:24.853256+0000","last_unstale":"2023-07-06T01:20:25.886696+0000","last_undegraded":"2023-07-06T01:20:25.886696+0000","last_fullsized":"2023-07-06T01:20:25.886696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:20.409326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884398+0000","last_change":"2023-07-06T01:20:24.852693+0000","last_active":"2023-07-06T01:20:25.884398+0000","last_peered":"2023-07-06T01:20:25.884398+0000","last_clean":"2023-07-06T01:20:25.884398+0000","last_became_active":"2023-07-06T01:20:24.852395+0000","last_became_peered":"2023-07-06T01:20:24.852395+0000","last_unstale":"2023-07-06T01:20:25.884398+0000","last_undegraded":"2023-07-06T01:20:25.884398+0000","last_fullsized":"2023-07-06T01:20:25.884398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:35.585237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836707+0000","last_change":"2023-07-06T01:20:24.856089+0000","last_active":"2023-07-06T01:20:25.836707+0000","last_peered":"2023-07-06T01:20:25.836707+0000","last_clean":"2023-07-06T01:20:25.836707+0000","last_became_active":"2023-07-06T01:20:24.855834+0000","last_became_peered":"2023-07-06T01:20:24.855834+0000","last_unstale":"2023-07-06T01:20:25.836707+0000","last_undegraded":"2023-07-06T01:20:25.836707+0000","last_fullsized":"2023-07-06T01:20:25.836707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:03.093141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859016+0000","last_change":"2023-07-06T01:20:24.856754+0000","last_active":"2023-07-06T01:20:24.859016+0000","last_peered":"2023-07-06T01:20:24.859016+0000","last_clean":"2023-07-06T01:20:24.859016+0000","last_became_active":"2023-07-06T01:20:24.856396+0000","last_became_peered":"2023-07-06T01:20:24.856396+0000","last_unstale":"2023-07-06T01:20:24.859016+0000","last_undegraded":"2023-07-06T01:20:24.859016+0000","last_fullsized":"2023-07-06T01:20:24.859016+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:06:02.668449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835952+0000","last_change":"2023-07-06T01:20:24.854996+0000","last_active":"2023-07-06T01:20:25.835952+0000","last_peered":"2023-07-06T01:20:25.835952+0000","last_clean":"2023-07-06T01:20:25.835952+0000","last_became_active":"2023-07-06T01:20:24.854737+0000","last_became_peered":"2023-07-06T01:20:24.854737+0000","last_unstale":"2023-07-06T01:20:25.835952+0000","last_undegraded":"2023-07-06T01:20:25.835952+0000","last_fullsized":"2023-07-06T01:20:25.835952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:51.766923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886641+0000","last_change":"2023-07-06T01:20:24.856804+0000","last_active":"2023-07-06T01:20:25.886641+0000","last_peered":"2023-07-06T01:20:25.886641+0000","last_clean":"2023-07-06T01:20:25.886641+0000","last_became_active":"2023-07-06T01:20:24.856490+0000","last_became_peered":"2023-07-06T01:20:24.856490+0000","last_unstale":"2023-07-06T01:20:25.886641+0000","last_undegraded":"2023-07-06T01:20:25.886641+0000","last_fullsized":"2023-07-06T01:20:25.886641+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:23:07.815642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835630+0000","last_change":"2023-07-06T01:20:24.850799+0000","last_active":"2023-07-06T01:20:25.835630+0000","last_peered":"2023-07-06T01:20:25.835630+0000","last_clean":"2023-07-06T01:20:25.835630+0000","last_became_active":"2023-07-06T01:20:24.850463+0000","last_became_peered":"2023-07-06T01:20:24.850463+0000","last_unstale":"2023-07-06T01:20:25.835630+0000","last_undegraded":"2023-07-06T01:20:25.835630+0000","last_fullsized":"2023-07-06T01:20:25.835630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:50.421447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835944+0000","last_change":"2023-07-06T01:20:24.857969+0000","last_active":"2023-07-06T01:20:25.835944+0000","last_peered":"2023-07-06T01:20:25.835944+0000","last_clean":"2023-07-06T01:20:25.835944+0000","last_became_active":"2023-07-06T01:20:24.857674+0000","last_became_peered":"2023-07-06T01:20:24.857674+0000","last_unstale":"2023-07-06T01:20:25.835944+0000","last_undegraded":"2023-07-06T01:20:25.835944+0000","last_fullsized":"2023-07-06T01:20:25.835944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:57.090630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.856018+0000","last_change":"2023-07-06T01:20:24.854034+0000","last_active":"2023-07-06T01:20:24.856018+0000","last_peered":"2023-07-06T01:20:24.856018+0000","last_clean":"2023-07-06T01:20:24.856018+0000","last_became_active":"2023-07-06T01:20:24.853820+0000","last_became_peered":"2023-07-06T01:20:24.853820+0000","last_unstale":"2023-07-06T01:20:24.856018+0000","last_undegraded":"2023-07-06T01:20:24.856018+0000","last_fullsized":"2023-07-06T01:20:24.856018+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:57.090630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836884+0000","last_change":"2023-07-06T01:20:23.849203+0000","last_active":"2023-07-06T01:20:25.836884+0000","last_peered":"2023-07-06T01:20:25.836884+0000","last_clean":"2023-07-06T01:20:25.836884+0000","last_became_active":"2023-07-06T01:20:23.848883+0000","last_became_peered":"2023-07-06T01:20:23.848883+0000","last_unstale":"2023-07-06T01:20:25.836884+0000","last_undegraded":"2023-07-06T01:20:25.836884+0000","last_fullsized":"2023-07-06T01:20:25.836884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:06.770007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.860470+0000","last_change":"2023-07-06T01:20:24.859359+0000","last_active":"2023-07-06T01:20:24.860470+0000","last_peered":"2023-07-06T01:20:24.860470+0000","last_clean":"2023-07-06T01:20:24.860470+0000","last_became_active":"2023-07-06T01:20:24.859152+0000","last_became_peered":"2023-07-06T01:20:24.859152+0000","last_unstale":"2023-07-06T01:20:24.860470+0000","last_undegraded":"2023-07-06T01:20:24.860470+0000","last_fullsized":"2023-07-06T01:20:24.860470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:01.181469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834689+0000","last_change":"2023-07-06T01:20:23.845980+0000","last_active":"2023-07-06T01:20:25.834689+0000","last_peered":"2023-07-06T01:20:25.834689+0000","last_clean":"2023-07-06T01:20:25.834689+0000","last_became_active":"2023-07-06T01:20:23.845696+0000","last_became_peered":"2023-07-06T01:20:23.845696+0000","last_unstale":"2023-07-06T01:20:25.834689+0000","last_undegraded":"2023-07-06T01:20:25.834689+0000","last_fullsized":"2023-07-06T01:20:25.834689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:58.579906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.858947+0000","last_change":"2023-07-06T01:20:23.846742+0000","last_active":"2023-07-06T01:20:24.858947+0000","last_peered":"2023-07-06T01:20:24.858947+0000","last_clean":"2023-07-06T01:20:24.858947+0000","last_became_active":"2023-07-06T01:20:23.846440+0000","last_became_peered":"2023-07-06T01:20:23.846440+0000","last_unstale":"2023-07-06T01:20:24.858947+0000","last_undegraded":"2023-07-06T01:20:24.858947+0000","last_fullsized":"2023-07-06T01:20:24.858947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:06.770007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862194+0000","last_change":"2023-07-06T01:20:24.861558+0000","last_active":"2023-07-06T01:20:24.862194+0000","last_peered":"2023-07-06T01:20:24.862194+0000","last_clean":"2023-07-06T01:20:24.862194+0000","last_became_active":"2023-07-06T01:20:24.860857+0000","last_became_peered":"2023-07-06T01:20:24.860857+0000","last_unstale":"2023-07-06T01:20:24.862194+0000","last_undegraded":"2023-07-06T01:20:24.862194+0000","last_fullsized":"2023-07-06T01:20:24.862194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:52.466998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.827224+0000","last_change":"2023-07-06T01:20:22.827224+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.827224+0000","last_undegraded":"2023-07-06T01:20:22.827224+0000","last_fullsized":"2023-07-06T01:20:22.827224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834373+0000","last_change":"2023-07-06T01:20:24.839527+0000","last_active":"2023-07-06T01:20:25.834373+0000","last_peered":"2023-07-06T01:20:25.834373+0000","last_clean":"2023-07-06T01:20:25.834373+0000","last_became_active":"2023-07-06T01:20:24.839201+0000","last_became_peered":"2023-07-06T01:20:24.839201+0000","last_unstale":"2023-07-06T01:20:25.834373+0000","last_undegraded":"2023-07-06T01:20:25.834373+0000","last_fullsized":"2023-07-06T01:20:25.834373+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:57:20.290658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886526+0000","last_change":"2023-07-06T01:20:23.853509+0000","last_active":"2023-07-06T01:20:25.886526+0000","last_peered":"2023-07-06T01:20:25.886526+0000","last_clean":"2023-07-06T01:20:25.886526+0000","last_became_active":"2023-07-06T01:20:23.853060+0000","last_became_peered":"2023-07-06T01:20:23.853060+0000","last_unstale":"2023-07-06T01:20:25.886526+0000","last_undegraded":"2023-07-06T01:20:25.886526+0000","last_fullsized":"2023-07-06T01:20:25.886526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:40.214108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886380+0000","last_change":"2023-07-06T01:20:24.856240+0000","last_active":"2023-07-06T01:20:25.886380+0000","last_peered":"2023-07-06T01:20:25.886380+0000","last_clean":"2023-07-06T01:20:25.886380+0000","last_became_active":"2023-07-06T01:20:24.853426+0000","last_became_peered":"2023-07-06T01:20:24.853426+0000","last_unstale":"2023-07-06T01:20:25.886380+0000","last_undegraded":"2023-07-06T01:20:25.886380+0000","last_fullsized":"2023-07-06T01:20:25.886380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:59.584568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.829932+0000","last_change":"2023-07-06T01:20:22.829932+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.829932+0000","last_undegraded":"2023-07-06T01:20:22.829932+0000","last_fullsized":"2023-07-06T01:20:22.829932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836572+0000","last_change":"2023-07-06T01:20:23.849040+0000","last_active":"2023-07-06T01:20:24.836572+0000","last_peered":"2023-07-06T01:20:24.836572+0000","last_clean":"2023-07-06T01:20:24.836572+0000","last_became_active":"2023-07-06T01:20:23.848756+0000","last_became_peered":"2023-07-06T01:20:23.848756+0000","last_unstale":"2023-07-06T01:20:24.836572+0000","last_undegraded":"2023-07-06T01:20:24.836572+0000","last_fullsized":"2023-07-06T01:20:24.836572+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:05:00.225120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855846+0000","last_change":"2023-07-06T01:20:24.849751+0000","last_active":"2023-07-06T01:20:24.855846+0000","last_peered":"2023-07-06T01:20:24.855846+0000","last_clean":"2023-07-06T01:20:24.855846+0000","last_became_active":"2023-07-06T01:20:24.849475+0000","last_became_peered":"2023-07-06T01:20:24.849475+0000","last_unstale":"2023-07-06T01:20:24.855846+0000","last_undegraded":"2023-07-06T01:20:24.855846+0000","last_fullsized":"2023-07-06T01:20:24.855846+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.826762+0000","last_change":"2023-07-06T01:20:22.826762+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.826762+0000","last_undegraded":"2023-07-06T01:20:22.826762+0000","last_fullsized":"2023-07-06T01:20:22.826762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884424+0000","last_change":"2023-07-06T01:20:24.860540+0000","last_active":"2023-07-06T01:20:25.884424+0000","last_peered":"2023-07-06T01:20:25.884424+0000","last_clean":"2023-07-06T01:20:25.884424+0000","last_became_active":"2023-07-06T01:20:24.859869+0000","last_became_peered":"2023-07-06T01:20:24.859869+0000","last_unstale":"2023-07-06T01:20:25.884424+0000","last_undegraded":"2023-07-06T01:20:25.884424+0000","last_fullsized":"2023-07-06T01:20:25.884424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:23.281099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.845141+0000","last_change":"2023-07-06T01:20:23.848957+0000","last_active":"2023-07-06T01:20:24.845141+0000","last_peered":"2023-07-06T01:20:24.845141+0000","last_clean":"2023-07-06T01:20:24.845141+0000","last_became_active":"2023-07-06T01:20:23.848646+0000","last_became_peered":"2023-07-06T01:20:23.848646+0000","last_unstale":"2023-07-06T01:20:24.845141+0000","last_undegraded":"2023-07-06T01:20:24.845141+0000","last_fullsized":"2023-07-06T01:20:24.845141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:35:50.683696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886834+0000","last_change":"2023-07-06T01:20:23.847858+0000","last_active":"2023-07-06T01:20:25.886834+0000","last_peered":"2023-07-06T01:20:25.886834+0000","last_clean":"2023-07-06T01:20:25.886834+0000","last_became_active":"2023-07-06T01:20:23.847549+0000","last_became_peered":"2023-07-06T01:20:23.847549+0000","last_unstale":"2023-07-06T01:20:25.886834+0000","last_undegraded":"2023-07-06T01:20:25.886834+0000","last_fullsized":"2023-07-06T01:20:25.886834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:53:56.761538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834565+0000","last_change":"2023-07-06T01:20:24.848275+0000","last_active":"2023-07-06T01:20:25.834565+0000","last_peered":"2023-07-06T01:20:25.834565+0000","last_clean":"2023-07-06T01:20:25.834565+0000","last_became_active":"2023-07-06T01:20:24.847949+0000","last_became_peered":"2023-07-06T01:20:24.847949+0000","last_unstale":"2023-07-06T01:20:25.834565+0000","last_undegraded":"2023-07-06T01:20:25.834565+0000","last_fullsized":"2023-07-06T01:20:25.834565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:24:03.775615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887068+0000","last_change":"2023-07-06T01:20:23.849878+0000","last_active":"2023-07-06T01:20:25.887068+0000","last_peered":"2023-07-06T01:20:25.887068+0000","last_clean":"2023-07-06T01:20:25.887068+0000","last_became_active":"2023-07-06T01:20:23.849573+0000","last_became_peered":"2023-07-06T01:20:23.849573+0000","last_unstale":"2023-07-06T01:20:25.887068+0000","last_undegraded":"2023-07-06T01:20:25.887068+0000","last_fullsized":"2023-07-06T01:20:25.887068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:12:26.209870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.868009+0000","last_change":"2023-07-06T01:20:23.845648+0000","last_active":"2023-07-06T01:20:24.868009+0000","last_peered":"2023-07-06T01:20:24.868009+0000","last_clean":"2023-07-06T01:20:24.868009+0000","last_became_active":"2023-07-06T01:20:23.845340+0000","last_became_peered":"2023-07-06T01:20:23.845340+0000","last_unstale":"2023-07-06T01:20:24.868009+0000","last_undegraded":"2023-07-06T01:20:24.868009+0000","last_fullsized":"2023-07-06T01:20:24.868009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:39:35.605373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834521+0000","last_change":"2023-07-06T01:20:24.858282+0000","last_active":"2023-07-06T01:20:25.834521+0000","last_peered":"2023-07-06T01:20:25.834521+0000","last_clean":"2023-07-06T01:20:25.834521+0000","last_became_active":"2023-07-06T01:20:24.857879+0000","last_became_peered":"2023-07-06T01:20:24.857879+0000","last_unstale":"2023-07-06T01:20:25.834521+0000","last_undegraded":"2023-07-06T01:20:25.834521+0000","last_fullsized":"2023-07-06T01:20:25.834521+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:22:46.902965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855788+0000","last_change":"2023-07-06T01:20:23.841745+0000","last_active":"2023-07-06T01:20:24.855788+0000","last_peered":"2023-07-06T01:20:24.855788+0000","last_clean":"2023-07-06T01:20:24.855788+0000","last_became_active":"2023-07-06T01:20:23.841438+0000","last_became_peered":"2023-07-06T01:20:23.841438+0000","last_unstale":"2023-07-06T01:20:24.855788+0000","last_undegraded":"2023-07-06T01:20:24.855788+0000","last_fullsized":"2023-07-06T01:20:24.855788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:54:11.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855590+0000","last_change":"2023-07-06T01:20:24.854272+0000","last_active":"2023-07-06T01:20:24.855590+0000","last_peered":"2023-07-06T01:20:24.855590+0000","last_clean":"2023-07-06T01:20:24.855590+0000","last_became_active":"2023-07-06T01:20:24.854002+0000","last_became_peered":"2023-07-06T01:20:24.854002+0000","last_unstale":"2023-07-06T01:20:24.855590+0000","last_undegraded":"2023-07-06T01:20:24.855590+0000","last_fullsized":"2023-07-06T01:20:24.855590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:23:07.815642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836795+0000","last_change":"2023-07-06T01:20:23.854212+0000","last_active":"2023-07-06T01:20:24.836795+0000","last_peered":"2023-07-06T01:20:24.836795+0000","last_clean":"2023-07-06T01:20:24.836795+0000","last_became_active":"2023-07-06T01:20:23.853928+0000","last_became_peered":"2023-07-06T01:20:23.853928+0000","last_unstale":"2023-07-06T01:20:24.836795+0000","last_undegraded":"2023-07-06T01:20:24.836795+0000","last_fullsized":"2023-07-06T01:20:24.836795+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:49.726864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835969+0000","last_change":"2023-07-06T01:20:23.847446+0000","last_active":"2023-07-06T01:20:25.835969+0000","last_peered":"2023-07-06T01:20:25.835969+0000","last_clean":"2023-07-06T01:20:25.835969+0000","last_became_active":"2023-07-06T01:20:23.846929+0000","last_became_peered":"2023-07-06T01:20:23.846929+0000","last_unstale":"2023-07-06T01:20:25.835969+0000","last_undegraded":"2023-07-06T01:20:25.835969+0000","last_fullsized":"2023-07-06T01:20:25.835969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:05:00.225120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855866+0000","last_change":"2023-07-06T01:20:24.855191+0000","last_active":"2023-07-06T01:20:24.855866+0000","last_peered":"2023-07-06T01:20:24.855866+0000","last_clean":"2023-07-06T01:20:24.855866+0000","last_became_active":"2023-07-06T01:20:24.854898+0000","last_became_peered":"2023-07-06T01:20:24.854898+0000","last_unstale":"2023-07-06T01:20:24.855866+0000","last_undegraded":"2023-07-06T01:20:24.855866+0000","last_fullsized":"2023-07-06T01:20:24.855866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.419832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.829961+0000","last_change":"2023-07-06T01:20:22.829961+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.829961+0000","last_undegraded":"2023-07-06T01:20:22.829961+0000","last_fullsized":"2023-07-06T01:20:22.829961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862166+0000","last_change":"2023-07-06T01:20:24.861072+0000","last_active":"2023-07-06T01:20:24.862166+0000","last_peered":"2023-07-06T01:20:24.862166+0000","last_clean":"2023-07-06T01:20:24.862166+0000","last_became_active":"2023-07-06T01:20:24.860745+0000","last_became_peered":"2023-07-06T01:20:24.860745+0000","last_unstale":"2023-07-06T01:20:24.862166+0000","last_undegraded":"2023-07-06T01:20:24.862166+0000","last_fullsized":"2023-07-06T01:20:24.862166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:45.220808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887003+0000","last_change":"2023-07-06T01:20:23.848801+0000","last_active":"2023-07-06T01:20:25.887003+0000","last_peered":"2023-07-06T01:20:25.887003+0000","last_clean":"2023-07-06T01:20:25.887003+0000","last_became_active":"2023-07-06T01:20:23.848412+0000","last_became_peered":"2023-07-06T01:20:23.848412+0000","last_unstale":"2023-07-06T01:20:25.887003+0000","last_undegraded":"2023-07-06T01:20:25.887003+0000","last_fullsized":"2023-07-06T01:20:25.887003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:35:50.683696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886952+0000","last_change":"2023-07-06T01:20:24.854853+0000","last_active":"2023-07-06T01:20:25.886952+0000","last_peered":"2023-07-06T01:20:25.886952+0000","last_clean":"2023-07-06T01:20:25.886952+0000","last_became_active":"2023-07-06T01:20:24.854159+0000","last_became_peered":"2023-07-06T01:20:24.854159+0000","last_unstale":"2023-07-06T01:20:25.886952+0000","last_undegraded":"2023-07-06T01:20:25.886952+0000","last_fullsized":"2023-07-06T01:20:25.886952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.419832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.826760+0000","last_change":"2023-07-06T01:20:22.826760+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.826760+0000","last_undegraded":"2023-07-06T01:20:22.826760+0000","last_fullsized":"2023-07-06T01:20:22.826760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884278+0000","last_change":"2023-07-06T01:20:23.846883+0000","last_active":"2023-07-06T01:20:25.884278+0000","last_peered":"2023-07-06T01:20:25.884278+0000","last_clean":"2023-07-06T01:20:25.884278+0000","last_became_active":"2023-07-06T01:20:23.846566+0000","last_became_peered":"2023-07-06T01:20:23.846566+0000","last_unstale":"2023-07-06T01:20:25.884278+0000","last_undegraded":"2023-07-06T01:20:25.884278+0000","last_fullsized":"2023-07-06T01:20:25.884278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:44:49.870273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884313+0000","last_change":"2023-07-06T01:20:24.853783+0000","last_active":"2023-07-06T01:20:25.884313+0000","last_peered":"2023-07-06T01:20:25.884313+0000","last_clean":"2023-07-06T01:20:25.884313+0000","last_became_active":"2023-07-06T01:20:24.853427+0000","last_became_peered":"2023-07-06T01:20:24.853427+0000","last_unstale":"2023-07-06T01:20:25.884313+0000","last_undegraded":"2023-07-06T01:20:25.884313+0000","last_fullsized":"2023-07-06T01:20:25.884313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:33:25.311596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.837321+0000","last_change":"2023-07-06T01:20:23.847016+0000","last_active":"2023-07-06T01:20:24.837321+0000","last_peered":"2023-07-06T01:20:24.837321+0000","last_clean":"2023-07-06T01:20:24.837321+0000","last_became_active":"2023-07-06T01:20:23.846748+0000","last_became_peered":"2023-07-06T01:20:23.846748+0000","last_unstale":"2023-07-06T01:20:24.837321+0000","last_undegraded":"2023-07-06T01:20:24.837321+0000","last_fullsized":"2023-07-06T01:20:24.837321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:13.424430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884268+0000","last_change":"2023-07-06T01:20:23.847882+0000","last_active":"2023-07-06T01:20:25.884268+0000","last_peered":"2023-07-06T01:20:25.884268+0000","last_clean":"2023-07-06T01:20:25.884268+0000","last_became_active":"2023-07-06T01:20:23.847534+0000","last_became_peered":"2023-07-06T01:20:23.847534+0000","last_unstale":"2023-07-06T01:20:25.884268+0000","last_undegraded":"2023-07-06T01:20:25.884268+0000","last_fullsized":"2023-07-06T01:20:25.884268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:22:12.321678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834687+0000","last_change":"2023-07-06T01:20:24.848033+0000","last_active":"2023-07-06T01:20:25.834687+0000","last_peered":"2023-07-06T01:20:25.834687+0000","last_clean":"2023-07-06T01:20:25.834687+0000","last_became_active":"2023-07-06T01:20:24.847514+0000","last_became_peered":"2023-07-06T01:20:24.847514+0000","last_unstale":"2023-07-06T01:20:25.834687+0000","last_undegraded":"2023-07-06T01:20:25.834687+0000","last_fullsized":"2023-07-06T01:20:25.834687+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:02.805903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884363+0000","last_change":"2023-07-06T01:20:23.850960+0000","last_active":"2023-07-06T01:20:25.884363+0000","last_peered":"2023-07-06T01:20:25.884363+0000","last_clean":"2023-07-06T01:20:25.884363+0000","last_became_active":"2023-07-06T01:20:23.850699+0000","last_became_peered":"2023-07-06T01:20:23.850699+0000","last_unstale":"2023-07-06T01:20:25.884363+0000","last_undegraded":"2023-07-06T01:20:25.884363+0000","last_fullsized":"2023-07-06T01:20:25.884363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:30:39.863076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886519+0000","last_change":"2023-07-06T01:20:24.861071+0000","last_active":"2023-07-06T01:20:25.886519+0000","last_peered":"2023-07-06T01:20:25.886519+0000","last_clean":"2023-07-06T01:20:25.886519+0000","last_became_active":"2023-07-06T01:20:24.860803+0000","last_became_peered":"2023-07-06T01:20:24.860803+0000","last_unstale":"2023-07-06T01:20:25.886519+0000","last_undegraded":"2023-07-06T01:20:25.886519+0000","last_fullsized":"2023-07-06T01:20:25.886519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:17.181181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.830814+0000","last_change":"2023-07-06T01:20:22.830814+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.830814+0000","last_undegraded":"2023-07-06T01:20:22.830814+0000","last_fullsized":"2023-07-06T01:20:22.830814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862183+0000","last_change":"2023-07-06T01:20:24.861226+0000","last_active":"2023-07-06T01:20:24.862183+0000","last_peered":"2023-07-06T01:20:24.862183+0000","last_clean":"2023-07-06T01:20:24.862183+0000","last_became_active":"2023-07-06T01:20:24.860944+0000","last_became_peered":"2023-07-06T01:20:24.860944+0000","last_unstale":"2023-07-06T01:20:24.862183+0000","last_undegraded":"2023-07-06T01:20:24.862183+0000","last_fullsized":"2023-07-06T01:20:24.862183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:12.103457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884125+0000","last_change":"2023-07-06T01:20:23.849860+0000","last_active":"2023-07-06T01:20:25.884125+0000","last_peered":"2023-07-06T01:20:25.884125+0000","last_clean":"2023-07-06T01:20:25.884125+0000","last_became_active":"2023-07-06T01:20:23.849507+0000","last_became_peered":"2023-07-06T01:20:23.849507+0000","last_unstale":"2023-07-06T01:20:25.884125+0000","last_undegraded":"2023-07-06T01:20:25.884125+0000","last_fullsized":"2023-07-06T01:20:25.884125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:51.413216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.826768+0000","last_change":"2023-07-06T01:20:22.826768+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.826768+0000","last_undegraded":"2023-07-06T01:20:22.826768+0000","last_fullsized":"2023-07-06T01:20:22.826768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T01:20:22.827804+0000","last_change":"2023-07-06T01:19:20.074195+0000","last_active":"2023-07-06T01:20:22.827804+0000","last_peered":"2023-07-06T01:20:22.827804+0000","last_clean":"2023-07-06T01:20:22.827804+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:20:22.827804+0000","last_undegraded":"2023-07-06T01:20:22.827804+0000","last_fullsized":"2023-07-06T01:20:22.827804+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:38:31.352462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.842093+0000","last_change":"2023-07-06T01:20:23.849570+0000","last_active":"2023-07-06T01:20:24.842093+0000","last_peered":"2023-07-06T01:20:24.842093+0000","last_clean":"2023-07-06T01:20:24.842093+0000","last_became_active":"2023-07-06T01:20:23.849255+0000","last_became_peered":"2023-07-06T01:20:23.849255+0000","last_unstale":"2023-07-06T01:20:24.842093+0000","last_undegraded":"2023-07-06T01:20:24.842093+0000","last_fullsized":"2023-07-06T01:20:24.842093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:53:56.761538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.846124+0000","last_change":"2023-07-06T01:19:19.587110+0000","last_active":"2023-07-06T01:20:24.846124+0000","last_peered":"2023-07-06T01:20:24.846124+0000","last_clean":"2023-07-06T01:20:24.846124+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:20:24.846124+0000","last_undegraded":"2023-07-06T01:20:24.846124+0000","last_fullsized":"2023-07-06T01:20:24.846124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:49:50.820241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859472+0000","last_change":"2023-07-06T01:20:24.858252+0000","last_active":"2023-07-06T01:20:24.859472+0000","last_peered":"2023-07-06T01:20:24.859472+0000","last_clean":"2023-07-06T01:20:24.859472+0000","last_became_active":"2023-07-06T01:20:24.856691+0000","last_became_peered":"2023-07-06T01:20:24.856691+0000","last_unstale":"2023-07-06T01:20:24.859472+0000","last_undegraded":"2023-07-06T01:20:24.859472+0000","last_fullsized":"2023-07-06T01:20:24.859472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:03.093141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887051+0000","last_change":"2023-07-06T01:20:23.847756+0000","last_active":"2023-07-06T01:20:25.887051+0000","last_peered":"2023-07-06T01:20:25.887051+0000","last_clean":"2023-07-06T01:20:25.887051+0000","last_became_active":"2023-07-06T01:20:23.847392+0000","last_became_peered":"2023-07-06T01:20:23.847392+0000","last_unstale":"2023-07-06T01:20:25.887051+0000","last_undegraded":"2023-07-06T01:20:25.887051+0000","last_fullsized":"2023-07-06T01:20:25.887051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:50:55.073152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":35,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836206+0000","last_change":"2023-07-06T01:19:19.584242+0000","last_active":"2023-07-06T01:20:24.836206+0000","last_peered":"2023-07-06T01:20:24.836206+0000","last_clean":"2023-07-06T01:20:24.836206+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:20:24.836206+0000","last_undegraded":"2023-07-06T01:20:24.836206+0000","last_fullsized":"2023-07-06T01:20:24.836206+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:02:18.883377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":183,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T01:20:22.831308+0000","last_change":"2023-07-06T01:19:19.586495+0000","last_active":"2023-07-06T01:20:22.831308+0000","last_peered":"2023-07-06T01:20:22.831308+0000","last_clean":"2023-07-06T01:20:22.831308+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:20:22.831308+0000","last_undegraded":"2023-07-06T01:20:22.831308+0000","last_fullsized":"2023-07-06T01:20:22.831308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:06:26.958597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-07-06T01:20:23.824595+0000","last_change":"2023-07-06T01:20:23.824595+0000","last_active":"2023-07-06T01:20:23.824595+0000","last_peered":"2023-07-06T01:20:23.824595+0000","last_clean":"2023-07-06T01:20:23.824595+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:23.824595+0000","last_undegraded":"2023-07-06T01:20:23.824595+0000","last_fullsized":"2023-07-06T01:20:23.824595+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.885927+0000","last_change":"2023-07-06T01:20:23.854385+0000","last_active":"2023-07-06T01:20:25.885927+0000","last_peered":"2023-07-06T01:20:25.885927+0000","last_clean":"2023-07-06T01:20:25.885927+0000","last_became_active":"2023-07-06T01:20:23.853982+0000","last_became_peered":"2023-07-06T01:20:23.853982+0000","last_unstale":"2023-07-06T01:20:25.885927+0000","last_undegraded":"2023-07-06T01:20:25.885927+0000","last_fullsized":"2023-07-06T01:20:25.885927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:01.772125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T01:20:22.831236+0000","last_change":"2023-07-06T01:19:19.586426+0000","last_active":"2023-07-06T01:20:22.831236+0000","last_peered":"2023-07-06T01:20:22.831236+0000","last_clean":"2023-07-06T01:20:22.831236+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:20:22.831236+0000","last_undegraded":"2023-07-06T01:20:22.831236+0000","last_fullsized":"2023-07-06T01:20:22.831236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:55:46.933540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886966+0000","last_change":"2023-07-06T01:20:24.855685+0000","last_active":"2023-07-06T01:20:25.886966+0000","last_peered":"2023-07-06T01:20:25.886966+0000","last_clean":"2023-07-06T01:20:25.886966+0000","last_became_active":"2023-07-06T01:20:24.855347+0000","last_became_peered":"2023-07-06T01:20:24.855347+0000","last_unstale":"2023-07-06T01:20:25.886966+0000","last_undegraded":"2023-07-06T01:20:25.886966+0000","last_fullsized":"2023-07-06T01:20:25.886966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:39:36.610034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-07-06T01:20:22.830379+0000","last_change":"2023-07-06T01:20:22.830379+0000","last_active":"2023-07-06T01:20:22.819933+0000","last_peered":"2023-07-06T01:20:22.819933+0000","last_clean":"2023-07-06T01:20:22.819933+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-07-06T01:20:22.830379+0000","last_undegraded":"2023-07-06T01:20:22.830379+0000","last_fullsized":"2023-07-06T01:20:22.830379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-07-06T01:20:22.831212+0000","last_change":"2023-07-06T01:19:19.586626+0000","last_active":"2023-07-06T01:20:22.831212+0000","last_peered":"2023-07-06T01:20:22.831212+0000","last_clean":"2023-07-06T01:20:22.831212+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:20:22.831212+0000","last_undegraded":"2023-07-06T01:20:22.831212+0000","last_fullsized":"2023-07-06T01:20:22.831212+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:06:51.308527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.852493+0000","last_change":"2023-07-06T01:20:24.845956+0000","last_active":"2023-07-06T01:20:24.852493+0000","last_peered":"2023-07-06T01:20:24.852493+0000","last_clean":"2023-07-06T01:20:24.852493+0000","last_became_active":"2023-07-06T01:20:24.845698+0000","last_became_peered":"2023-07-06T01:20:24.845698+0000","last_unstale":"2023-07-06T01:20:24.852493+0000","last_undegraded":"2023-07-06T01:20:24.852493+0000","last_fullsized":"2023-07-06T01:20:24.852493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:41.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834898+0000","last_change":"2023-07-06T01:20:23.847902+0000","last_active":"2023-07-06T01:20:25.834898+0000","last_peered":"2023-07-06T01:20:25.834898+0000","last_clean":"2023-07-06T01:20:25.834898+0000","last_became_active":"2023-07-06T01:20:23.847543+0000","last_became_peered":"2023-07-06T01:20:23.847543+0000","last_unstale":"2023-07-06T01:20:25.834898+0000","last_undegraded":"2023-07-06T01:20:25.834898+0000","last_fullsized":"2023-07-06T01:20:25.834898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:51.462337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886549+0000","last_change":"2023-07-06T01:19:19.584086+0000","last_active":"2023-07-06T01:20:25.886549+0000","last_peered":"2023-07-06T01:20:25.886549+0000","last_clean":"2023-07-06T01:20:25.886549+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:20:25.886549+0000","last_undegraded":"2023-07-06T01:20:25.886549+0000","last_fullsized":"2023-07-06T01:20:25.886549+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:01:33.579869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855022+0000","last_change":"2023-07-06T01:20:24.851662+0000","last_active":"2023-07-06T01:20:24.855022+0000","last_peered":"2023-07-06T01:20:24.855022+0000","last_clean":"2023-07-06T01:20:24.855022+0000","last_became_active":"2023-07-06T01:20:24.851349+0000","last_became_peered":"2023-07-06T01:20:24.851349+0000","last_unstale":"2023-07-06T01:20:24.855022+0000","last_undegraded":"2023-07-06T01:20:24.855022+0000","last_fullsized":"2023-07-06T01:20:24.855022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:50.421447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836073+0000","last_change":"2023-07-06T01:20:23.845197+0000","last_active":"2023-07-06T01:20:25.836073+0000","last_peered":"2023-07-06T01:20:25.836073+0000","last_clean":"2023-07-06T01:20:25.836073+0000","last_became_active":"2023-07-06T01:20:23.844508+0000","last_became_peered":"2023-07-06T01:20:23.844508+0000","last_unstale":"2023-07-06T01:20:25.836073+0000","last_undegraded":"2023-07-06T01:20:25.836073+0000","last_fullsized":"2023-07-06T01:20:25.836073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:00.176807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836633+0000","last_change":"2023-07-06T01:19:19.586424+0000","last_active":"2023-07-06T01:20:25.836633+0000","last_peered":"2023-07-06T01:20:25.836633+0000","last_clean":"2023-07-06T01:20:25.836633+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:20:25.836633+0000","last_undegraded":"2023-07-06T01:20:25.836633+0000","last_fullsized":"2023-07-06T01:20:25.836633+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:04:57.410877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834442+0000","last_change":"2023-07-06T01:20:24.849984+0000","last_active":"2023-07-06T01:20:25.834442+0000","last_peered":"2023-07-06T01:20:25.834442+0000","last_clean":"2023-07-06T01:20:25.834442+0000","last_became_active":"2023-07-06T01:20:24.849603+0000","last_became_peered":"2023-07-06T01:20:24.849603+0000","last_unstale":"2023-07-06T01:20:25.834442+0000","last_undegraded":"2023-07-06T01:20:25.834442+0000","last_fullsized":"2023-07-06T01:20:25.834442+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:39:50.052979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834826+0000","last_change":"2023-07-06T01:20:23.842452+0000","last_active":"2023-07-06T01:20:25.834826+0000","last_peered":"2023-07-06T01:20:25.834826+0000","last_clean":"2023-07-06T01:20:25.834826+0000","last_became_active":"2023-07-06T01:20:23.842176+0000","last_became_peered":"2023-07-06T01:20:23.842176+0000","last_unstale":"2023-07-06T01:20:25.834826+0000","last_undegraded":"2023-07-06T01:20:25.834826+0000","last_fullsized":"2023-07-06T01:20:25.834826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:16.176519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836914+0000","last_change":"2023-07-06T01:19:19.583699+0000","last_active":"2023-07-06T01:20:25.836914+0000","last_peered":"2023-07-06T01:20:25.836914+0000","last_clean":"2023-07-06T01:20:25.836914+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:20:25.836914+0000","last_undegraded":"2023-07-06T01:20:25.836914+0000","last_fullsized":"2023-07-06T01:20:25.836914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:52:09.171519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":94,"acting":94,"num_store_stats":6},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":925696,"data_stored":721739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.55100000000000005}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.505},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.439},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45200000000000001}]},{"osd":3,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53200000000000003}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.34100000000000003}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.375}]}]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":344064,"data_stored":137541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.499},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47599999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.436},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.39800000000000002}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.622}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":966656,"data_stored":741755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":12.23,"5min":12.23,"15min":12.23},"last":0.311},{"interface":"front","average":{"1min":1.839,"5min":1.839,"15min":1.839},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":15.208,"5min":15.208,"15min":15.208},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":12.074,"5min":12.074,"15min":12.074},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":11.988,"5min":11.988,"15min":11.988},"last":0.73999999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.173,"5min":2.173,"15min":2.173},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"last":0.442},{"interface":"front","average":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":15.244999999999999,"5min":15.244999999999999,"15min":15.244999999999999},"last":0.379}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":11.923,"5min":11.923,"15min":11.923},"last":2.6080000000000001},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":12.019,"5min":12.019,"15min":12.019},"last":3.5950000000000002}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":11.999000000000001,"5min":11.999000000000001,"15min":11.999000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":11.951000000000001,"5min":11.951000000000001,"15min":11.951000000000001},"last":0.624}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":15.167999999999999,"5min":15.167999999999999,"15min":15.167999999999999},"last":2.7490000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":12.041,"5min":12.041,"15min":12.041},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":15.115,"5min":15.115,"15min":15.115},"last":1.625},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":11.965,"5min":11.965,"15min":11.965},"last":1.8200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":157984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":0.251}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.215},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.27200000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.503},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":6.3049999999999997,"5min":6.3049999999999997,"15min":6.3049999999999997},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.2430000000000003,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.63900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.436},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.625}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54900000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":152895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.307},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.375}]},{"osd":2,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.32700000000000001}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":3.8130000000000002,"5min":3.8130000000000002,"15min":3.8130000000000002},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.71899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":158062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.5},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.436}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":158532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.629},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.505},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.439}]},{"osd":5,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42399999999999999}]},{"osd":7,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:20:29.798 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2023-07-06T01:20:29.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.799 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2023-07-06T01:20:29.799 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2023-07-06T01:20:29.800 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.800 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2023-07-06T01:20:29.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.801 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2023-07-06T01:20:29.801 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2023-07-06T01:20:29.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2023-07-06T01:20:29.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-07-06T01:20:29.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-07-06T01:20:29.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-07-06T01:20:22.827224+0000', 'last_change': '2023-07-06T01:20:22.827224+0000', 'last_active': '2023-07-06T01:20:22.819933+0000', 'last_peered': '2023-07-06T01:20:22.819933+0000', 'last_clean': '2023-07-06T01:20:22.819933+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:22.827224+0000', 'last_undegraded': '2023-07-06T01:20:22.827224+0000', 'last_fullsized': '2023-07-06T01:20:22.827224+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T01:20:29.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-07-06T01:20:29.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-07-06T01:20:22.829932+0000', 'last_change': '2023-07-06T01:20:22.829932+0000', 'last_active': '2023-07-06T01:20:22.819933+0000', 'last_peered': '2023-07-06T01:20:22.819933+0000', 'last_clean': '2023-07-06T01:20:22.819933+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:22.829932+0000', 'last_undegraded': '2023-07-06T01:20:22.829932+0000', 'last_fullsized': '2023-07-06T01:20:22.829932+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T01:20:29.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-07-06T01:20:29.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-07-06T01:20:29.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-07-06T01:20:22.826762+0000', 'last_change': '2023-07-06T01:20:22.826762+0000', 'last_active': '2023-07-06T01:20:22.819933+0000', 'last_peered': '2023-07-06T01:20:22.819933+0000', 'last_clean': '2023-07-06T01:20:22.819933+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:22.826762+0000', 'last_undegraded': '2023-07-06T01:20:22.826762+0000', 'last_fullsized': '2023-07-06T01:20:22.826762+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T01:20:29.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2023-07-06T01:20:29.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-07-06T01:20:29.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2023-07-06T01:20:29.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-07-06T01:20:29.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-07-06T01:20:22.829961+0000', 'last_change': '2023-07-06T01:20:22.829961+0000', 'last_active': '2023-07-06T01:20:22.819933+0000', 'last_peered': '2023-07-06T01:20:22.819933+0000', 'last_clean': '2023-07-06T01:20:22.819933+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:22.829961+0000', 'last_undegraded': '2023-07-06T01:20:22.829961+0000', 'last_fullsized': '2023-07-06T01:20:22.829961+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T01:20:29.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-07-06T01:20:29.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-07-06T01:20:22.826760+0000', 'last_change': '2023-07-06T01:20:22.826760+0000', 'last_active': '2023-07-06T01:20:22.819933+0000', 'last_peered': '2023-07-06T01:20:22.819933+0000', 'last_clean': '2023-07-06T01:20:22.819933+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:22.826760+0000', 'last_undegraded': '2023-07-06T01:20:22.826760+0000', 'last_fullsized': '2023-07-06T01:20:22.826760+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T01:20:29.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-07-06T01:20:29.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2023-07-06T01:20:29.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-07-06T01:20:29.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-07-06T01:20:22.830814+0000', 'last_change': '2023-07-06T01:20:22.830814+0000', 'last_active': '2023-07-06T01:20:22.819933+0000', 'last_peered': '2023-07-06T01:20:22.819933+0000', 'last_clean': '2023-07-06T01:20:22.819933+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:22.830814+0000', 'last_undegraded': '2023-07-06T01:20:22.830814+0000', 'last_fullsized': '2023-07-06T01:20:22.830814+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T01:20:29.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-07-06T01:20:29.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-07-06T01:20:29.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-07-06T01:20:22.826768+0000', 'last_change': '2023-07-06T01:20:22.826768+0000', 'last_active': '2023-07-06T01:20:22.819933+0000', 'last_peered': '2023-07-06T01:20:22.819933+0000', 'last_clean': '2023-07-06T01:20:22.819933+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:22.826768+0000', 'last_undegraded': '2023-07-06T01:20:22.826768+0000', 'last_fullsized': '2023-07-06T01:20:22.826768+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-07-06T01:20:29.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2023-07-06T01:20:29.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2023-07-06T01:20:29.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-07-06T01:20:23.824595+0000', 'last_change': '2023-07-06T01:20:23.824595+0000', 'last_active': '2023-07-06T01:20:23.824595+0000', 'last_peered': '2023-07-06T01:20:23.824595+0000', 'last_clean': '2023-07-06T01:20:23.824595+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:23.824595+0000', 'last_undegraded': '2023-07-06T01:20:23.824595+0000', 'last_fullsized': '2023-07-06T01:20:23.824595+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:23.824595+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-07-06T01:20:29.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-07-06T01:20:29.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-07-06T01:20:22.830379+0000', 'last_change': '2023-07-06T01:20:22.830379+0000', 'last_active': '2023-07-06T01:20:22.819933+0000', 'last_peered': '2023-07-06T01:20:22.819933+0000', 'last_clean': '2023-07-06T01:20:22.819933+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-07-06T01:20:22.830379+0000', 'last_undegraded': '2023-07-06T01:20:22.830379+0000', 'last_fullsized': '2023-07-06T01:20:22.830379+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'last_clean_scrub_stamp': '2023-07-06T01:20:22.819933+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-07-06T01:20:29.815 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-07-06T01:20:29.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.955+0000 7f1c43c19700 1 Processor -- start 2023-07-06T01:20:29.957 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.955+0000 7f1c43c19700 1 -- start start 2023-07-06T01:20:29.958 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.956+0000 7f1c43c19700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1c3c15a710 0x7f1c3c0ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.956+0000 7f1c43c19700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f1c3c0aecb0 0x7f1c3c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.956+0000 7f1c43c19700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c3c0ceda0 0x7f1c3c15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.956+0000 7f1c43c19700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3c0cf220 con 0x7f1c3c15a710 2023-07-06T01:20:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.956+0000 7f1c43c19700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3c0af270 con 0x7f1c3c0aecb0 2023-07-06T01:20:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.956+0000 7f1c43c19700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c0af550 con 0x7f1c3c0ceda0 2023-07-06T01:20:29.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.956+0000 7f1c419b5700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1c3c15a710 0x7f1c3c0ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.956+0000 7f1c419b5700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1c3c15a710 0x7f1c3c0ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47468/0 (socket says 172.21.15.22:47468) 2023-07-06T01:20:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.956+0000 7f1c419b5700 1 -- 172.21.15.22:0/871164497 learned_addr learned my addr 172.21.15.22:0/871164497 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.957+0000 7f1c411b4700 1 --2- 172.21.15.22:0/871164497 >> v2:172.21.15.103:3300/0 conn(0x7f1c3c0aecb0 0x7f1c3c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.957+0000 7f1c421b6700 1 --2- 172.21.15.22:0/871164497 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c3c0ceda0 0x7f1c3c15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.957+0000 7f1c419b5700 1 -- 172.21.15.22:0/871164497 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c3c0ceda0 msgr2=0x7f1c3c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.957+0000 7f1c419b5700 1 --2- 172.21.15.22:0/871164497 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c3c0ceda0 0x7f1c3c15e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.957+0000 7f1c419b5700 1 -- 172.21.15.22:0/871164497 >> v2:172.21.15.103:3300/0 conn(0x7f1c3c0aecb0 msgr2=0x7f1c3c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.957+0000 7f1c419b5700 1 --2- 172.21.15.22:0/871164497 >> v2:172.21.15.103:3300/0 conn(0x7f1c3c0aecb0 0x7f1c3c0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.957+0000 7f1c419b5700 1 -- 172.21.15.22:0/871164497 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c0af830 con 0x7f1c3c15a710 2023-07-06T01:20:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.957+0000 7f1c419b5700 1 --2- 172.21.15.22:0/871164497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c3c15a710 0x7f1c3c0ae770 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c071ac0 tx=0x7f1c38009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f716566e5ab35707 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.957+0000 7f1c409b3700 1 -- 172.21.15.22:0/871164497 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800b870 con 0x7f1c3c15a710 2023-07-06T01:20:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.957+0000 7f1c409b3700 1 -- 172.21.15.22:0/871164497 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3800b9d0 con 0x7f1c3c15a710 2023-07-06T01:20:29.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.957+0000 7f1c409b3700 1 -- 172.21.15.22:0/871164497 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800bd60 con 0x7f1c3c15a710 2023-07-06T01:20:29.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.957+0000 7f1c43c19700 1 -- 172.21.15.22:0/871164497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c3c15a710 msgr2=0x7f1c3c0ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.958+0000 7f1c43c19700 1 --2- 172.21.15.22:0/871164497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c3c15a710 0x7f1c3c0ae770 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c071ac0 tx=0x7f1c38009bd0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.958+0000 7f1c43c19700 1 -- 172.21.15.22:0/871164497 shutdown_connections 2023-07-06T01:20:29.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.958+0000 7f1c43c19700 1 --2- 172.21.15.22:0/871164497 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c3c0ceda0 0x7f1c3c15e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.958+0000 7f1c43c19700 1 --2- 172.21.15.22:0/871164497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c3c15a710 0x7f1c3c0ae770 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.958+0000 7f1c43c19700 1 --2- 172.21.15.22:0/871164497 >> v2:172.21.15.103:3300/0 conn(0x7f1c3c0aecb0 0x7f1c3c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.958+0000 7f1c43c19700 1 -- 172.21.15.22:0/871164497 >> 172.21.15.22:0/871164497 conn(0x7f1c3c0999f0 msgr2=0x7f1c3c099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:29.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.958+0000 7f1c43c19700 1 -- 172.21.15.22:0/871164497 shutdown_connections 2023-07-06T01:20:29.964 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.958+0000 7f1c43c19700 1 -- 172.21.15.22:0/871164497 wait complete. 2023-07-06T01:20:29.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.959+0000 7f1c43c19700 1 Processor -- start 2023-07-06T01:20:29.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.959+0000 7f1c43c19700 1 -- start start 2023-07-06T01:20:29.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c43c19700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c3c0aecb0 0x7f1c3c1f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c43c19700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c3c0ceda0 0x7f1c3c0c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c43c19700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c3c15a710 0x7f1c3c0ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c43c19700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c14fb70 con 0x7f1c3c0aecb0 2023-07-06T01:20:29.965 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c43c19700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c06c6e0 con 0x7f1c3c0ceda0 2023-07-06T01:20:29.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c43c19700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c06c9c0 con 0x7f1c3c15a710 2023-07-06T01:20:29.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c419b5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c3c0aecb0 0x7f1c3c1f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c419b5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c3c0aecb0 0x7f1c3c1f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47474/0 (socket says 172.21.15.22:47474) 2023-07-06T01:20:29.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c419b5700 1 -- 172.21.15.22:0/1422777596 learned_addr learned my addr 172.21.15.22:0/1422777596 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:29.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c411b4700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c3c0ceda0 0x7f1c3c0c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c421b6700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c3c15a710 0x7f1c3c0ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c419b5700 1 -- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c3c15a710 msgr2=0x7f1c3c0ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.966 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c419b5700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c3c15a710 0x7f1c3c0ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c419b5700 1 -- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c3c0ceda0 msgr2=0x7f1c3c0c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:29.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c419b5700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c3c0ceda0 0x7f1c3c0c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:29.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c419b5700 1 -- 172.21.15.22:0/1422777596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c38009550 con 0x7f1c3c0aecb0 2023-07-06T01:20:29.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c421b6700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c3c15a710 0x7f1c3c0ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:29.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c419b5700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c3c0aecb0 0x7f1c3c1f1c60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1c38014040 tx=0x7f1c38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:29.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c2e7fc700 1 -- 172.21.15.22:0/1422777596 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38004310 con 0x7f1c3c0aecb0 2023-07-06T01:20:29.967 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c2e7fc700 1 -- 172.21.15.22:0/1422777596 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3801e070 con 0x7f1c3c0aecb0 2023-07-06T01:20:29.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c2e7fc700 1 -- 172.21.15.22:0/1422777596 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c380036a0 con 0x7f1c3c0aecb0 2023-07-06T01:20:29.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c43c19700 1 -- 172.21.15.22:0/1422777596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3c14fe30 con 0x7f1c3c0aecb0 2023-07-06T01:20:29.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.960+0000 7f1c43c19700 1 -- 172.21.15.22:0/1422777596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3c1f21a0 con 0x7f1c3c0aecb0 2023-07-06T01:20:29.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.961+0000 7f1c43c19700 1 -- 172.21.15.22:0/1422777596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3c150350 con 0x7f1c3c0aecb0 2023-07-06T01:20:29.968 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.962+0000 7f1c2e7fc700 1 -- 172.21.15.22:0/1422777596 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1c38020030 con 0x7f1c3c0aecb0 2023-07-06T01:20:29.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.962+0000 7f1c2e7fc700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1c2006ace0 0x7f1c2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:29.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.962+0000 7f1c2e7fc700 1 -- 172.21.15.22:0/1422777596 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1c3808a320 con 0x7f1c3c0aecb0 2023-07-06T01:20:29.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.962+0000 7f1c411b4700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1c2006ace0 0x7f1c2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:29.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.963+0000 7f1c411b4700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1c2006ace0 0x7f1c2006d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c1f5840 tx=0x7f1c30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:29.969 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:29.967+0000 7f1c2e7fc700 1 -- 172.21.15.22:0/1422777596 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1c3c150350 con 0x7f1c3c0aecb0 2023-07-06T01:20:30.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.245+0000 7f1c43c19700 1 -- 172.21.15.22:0/1422777596 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1c3c150350 con 0x7f1c3c0aecb0 2023-07-06T01:20:30.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.245+0000 7f1c2e7fc700 1 -- 172.21.15.22:0/1422777596 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1722 (secure 0 0 0) 0x7f1c3c150350 con 0x7f1c3c0aecb0 2023-07-06T01:20:30.248 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:30.248 INFO:teuthology.orchestra.run.smithi022.stdout:{"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":83,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1688606357,"num_in_osds":8,"osd_in_since":1688606348,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":80},{"state_name":"unknown","count":17},{"state_name":"creating+peering","count":8}],"num_pgs":105,"num_pools":4,"num_objects":18,"data_bytes":592730,"bytes_used":224956416,"bytes_avail":772869156864,"bytes_total":773094113280,"unknown_pgs_ratio":0.16190476715564728,"inactive_pgs_ratio":0.076190479099750519,"write_bytes_sec":2762,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5115}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-07-06T01:20:20.012489+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-07-06T01:20:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.247+0000 7f1c43c19700 1 -- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1c2006ace0 msgr2=0x7f1c2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.247+0000 7f1c43c19700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1c2006ace0 0x7f1c2006d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c1f5840 tx=0x7f1c30008040 comp rx=0 tx=0).stop 2023-07-06T01:20:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.247+0000 7f1c43c19700 1 -- 172.21.15.22:0/1422777596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c3c0aecb0 msgr2=0x7f1c3c1f1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.247+0000 7f1c43c19700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c3c0aecb0 0x7f1c3c1f1c60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1c38014040 tx=0x7f1c38004750 comp rx=0 tx=0).stop 2023-07-06T01:20:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.247+0000 7f1c43c19700 1 -- 172.21.15.22:0/1422777596 shutdown_connections 2023-07-06T01:20:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.247+0000 7f1c43c19700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c3c15a710 0x7f1c3c0ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.247+0000 7f1c43c19700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c3c0aecb0 0x7f1c3c1f1c60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.247+0000 7f1c43c19700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1c2006ace0 0x7f1c2006d190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.247+0000 7f1c43c19700 1 --2- 172.21.15.22:0/1422777596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c3c0ceda0 0x7f1c3c0c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.247+0000 7f1c43c19700 1 -- 172.21.15.22:0/1422777596 >> 172.21.15.22:0/1422777596 conn(0x7f1c3c0999f0 msgr2=0x7f1c3c0ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.247+0000 7f1c43c19700 1 -- 172.21.15.22:0/1422777596 shutdown_connections 2023-07-06T01:20:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.247+0000 7f1c43c19700 1 -- 172.21.15.22:0/1422777596 wait complete. 2023-07-06T01:20:30.261 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-07-06T01:20:30.262 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:20:30.386 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.385+0000 7f66e17da700 1 Processor -- start 2023-07-06T01:20:30.387 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.385+0000 7f66e17da700 1 -- start start 2023-07-06T01:20:30.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66e17da700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f66dc0b36f0 0x7f66dc0cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:30.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66e17da700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66dc0d11a0 0x7f66dc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:30.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66e17da700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66dc0cebd0 con 0x7f66dc0cf5a0 2023-07-06T01:20:30.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66e17da700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66dc15cff0 con 0x7f66dc0b36f0 2023-07-06T01:20:30.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66e17da700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc0cdfc0 con 0x7f66dc0d11a0 2023-07-06T01:20:30.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66daffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f66dc0b36f0 0x7f66dc0cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:30.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66da7fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f66dc0cf5a0 0x7f66dc0d0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38226/0 (socket says 172.21.15.22:38226) 2023-07-06T01:20:30.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66da7fc700 1 -- 172.21.15.22:0/728038261 learned_addr learned my addr 172.21.15.22:0/728038261 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:30.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66db7fe700 1 --2- 172.21.15.22:0/728038261 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66dc0d11a0 0x7f66dc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:30.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66daffd700 1 -- 172.21.15.22:0/728038261 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66dc0d11a0 msgr2=0x7f66dc0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:30.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66daffd700 1 --2- 172.21.15.22:0/728038261 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66dc0d11a0 0x7f66dc0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:30.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66daffd700 1 -- 172.21.15.22:0/728038261 >> v1:172.21.15.22:6789/0 conn(0x7f66dc0cf5a0 legacy=0x7f66dc0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:30.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66daffd700 1 -- 172.21.15.22:0/728038261 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc0ce2a0 con 0x7f66dc0b36f0 2023-07-06T01:20:30.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66d9ffb700 1 -- 172.21.15.22:0/728038261 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623161193 0 0) 0x7f66dc0cebd0 con 0x7f66dc0cf5a0 2023-07-06T01:20:30.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66daffd700 1 --2- 172.21.15.22:0/728038261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66dc0b36f0 0x7f66dc0cf060 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009840 tx=0x7f66cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7147795a543240 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:30.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66d9ffb700 1 -- 172.21.15.22:0/728038261 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc011830 con 0x7f66dc0b36f0 2023-07-06T01:20:30.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66d9ffb700 1 -- 172.21.15.22:0/728038261 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc011990 con 0x7f66dc0b36f0 2023-07-06T01:20:30.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.386+0000 7f66d9ffb700 1 -- 172.21.15.22:0/728038261 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc011d20 con 0x7f66dc0b36f0 2023-07-06T01:20:30.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.387+0000 7f66e17da700 1 -- 172.21.15.22:0/728038261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66dc0b36f0 msgr2=0x7f66dc0cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:30.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.387+0000 7f66e17da700 1 --2- 172.21.15.22:0/728038261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66dc0b36f0 0x7f66dc0cf060 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009840 tx=0x7f66cc009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:30.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.387+0000 7f66e17da700 1 -- 172.21.15.22:0/728038261 shutdown_connections 2023-07-06T01:20:30.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.387+0000 7f66e17da700 1 --2- 172.21.15.22:0/728038261 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66dc0d11a0 0x7f66dc0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:30.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.387+0000 7f66e17da700 1 --2- 172.21.15.22:0/728038261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66dc0b36f0 0x7f66dc0cf060 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:30.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.387+0000 7f66e17da700 1 -- 172.21.15.22:0/728038261 >> 172.21.15.22:0/728038261 conn(0x7f66dc09c2e0 msgr2=0x7f66dc0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:30.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.387+0000 7f66e17da700 1 -- 172.21.15.22:0/728038261 shutdown_connections 2023-07-06T01:20:30.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.387+0000 7f66e17da700 1 -- 172.21.15.22:0/728038261 wait complete. 2023-07-06T01:20:30.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.389+0000 7f66e17da700 1 Processor -- start 2023-07-06T01:20:30.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.389+0000 7f66e17da700 1 -- start start 2023-07-06T01:20:30.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66e17da700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66dc0b36f0 0x7f66dc1f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:30.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66e17da700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66dc0cf5a0 0x7f66dc15e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:30.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66e17da700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66dc0d11a0 0x7f66dc0b8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66e17da700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc0cca50 con 0x7f66dc0b36f0 2023-07-06T01:20:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66e17da700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc0ccd30 con 0x7f66dc0d11a0 2023-07-06T01:20:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66e17da700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc0ca030 con 0x7f66dc0cf5a0 2023-07-06T01:20:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66daffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66dc0b36f0 0x7f66dc1f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66daffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66dc0b36f0 0x7f66dc1f2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47488/0 (socket says 172.21.15.22:47488) 2023-07-06T01:20:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66daffd700 1 -- 172.21.15.22:0/1536862664 learned_addr learned my addr 172.21.15.22:0/1536862664 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66db7fe700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66dc0d11a0 0x7f66dc0b8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:30.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66da7fc700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66dc0cf5a0 0x7f66dc15e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:30.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66daffd700 1 -- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66dc0cf5a0 msgr2=0x7f66dc15e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:30.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66daffd700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66dc0cf5a0 0x7f66dc15e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:30.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66daffd700 1 -- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66dc0d11a0 msgr2=0x7f66dc0b8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:30.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66daffd700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66dc0d11a0 0x7f66dc0b8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:30.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66daffd700 1 -- 172.21.15.22:0/1536862664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc0094a0 con 0x7f66dc0b36f0 2023-07-06T01:20:30.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66db7fe700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66dc0d11a0 0x7f66dc0b8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:30.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66daffd700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66dc0b36f0 0x7f66dc1f2b50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f66cc000c00 tx=0x7f66cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:30.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.390+0000 7f66c37fe700 1 -- 172.21.15.22:0/1536862664 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0045f0 con 0x7f66dc0b36f0 2023-07-06T01:20:30.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.391+0000 7f66c37fe700 1 -- 172.21.15.22:0/1536862664 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc01f070 con 0x7f66dc0b36f0 2023-07-06T01:20:30.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.391+0000 7f66e17da700 1 -- 172.21.15.22:0/1536862664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66dc0ca310 con 0x7f66dc0b36f0 2023-07-06T01:20:30.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.391+0000 7f66c37fe700 1 -- 172.21.15.22:0/1536862664 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc003ae0 con 0x7f66dc0b36f0 2023-07-06T01:20:30.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.391+0000 7f66e17da700 1 -- 172.21.15.22:0/1536862664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66dc1f3090 con 0x7f66dc0b36f0 2023-07-06T01:20:30.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.391+0000 7f66e17da700 1 -- 172.21.15.22:0/1536862664 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66dc0b4840 con 0x7f66dc0b36f0 2023-07-06T01:20:30.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.392+0000 7f66c37fe700 1 -- 172.21.15.22:0/1536862664 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f66cc003c40 con 0x7f66dc0b36f0 2023-07-06T01:20:30.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.392+0000 7f66c37fe700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f66c806ad10 0x7f66c806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:30.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.393+0000 7f66c37fe700 1 -- 172.21.15.22:0/1536862664 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f66cc021070 con 0x7f66dc0b36f0 2023-07-06T01:20:30.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.393+0000 7f66da7fc700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f66c806ad10 0x7f66c806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:30.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.393+0000 7f66da7fc700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f66c806ad10 0x7f66c806d1c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0a0e90 tx=0x7f66d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:30.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.397+0000 7f66c37fe700 1 -- 172.21.15.22:0/1536862664 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f66cc056020 con 0x7f66dc0b36f0 2023-07-06T01:20:30.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.588+0000 7f66e17da700 1 -- 172.21.15.22:0/1536862664 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f66dc0ce860 con 0x7f66c806ad10 2023-07-06T01:20:30.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.598+0000 7f66c37fe700 1 -- 172.21.15.22:0/1536862664 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311210 (secure 0 0 0) 0x7f66d00013d0 con 0x7f66c806ad10 2023-07-06T01:20:30.602 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:30.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.601+0000 7f66e17da700 1 -- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f66c806ad10 msgr2=0x7f66c806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:30.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.601+0000 7f66e17da700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f66c806ad10 0x7f66c806d1c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0a0e90 tx=0x7f66d0009040 comp rx=0 tx=0).stop 2023-07-06T01:20:30.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.601+0000 7f66e17da700 1 -- 172.21.15.22:0/1536862664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66dc0b36f0 msgr2=0x7f66dc1f2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:30.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.601+0000 7f66e17da700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66dc0b36f0 0x7f66dc1f2b50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f66cc000c00 tx=0x7f66cc004750 comp rx=0 tx=0).stop 2023-07-06T01:20:30.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.601+0000 7f66e17da700 1 -- 172.21.15.22:0/1536862664 shutdown_connections 2023-07-06T01:20:30.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.601+0000 7f66e17da700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f66c806ad10 0x7f66c806d1c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:30.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.601+0000 7f66e17da700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66dc0d11a0 0x7f66dc0b8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:30.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.601+0000 7f66e17da700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66dc0b36f0 0x7f66dc1f2b50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:30.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.601+0000 7f66e17da700 1 --2- 172.21.15.22:0/1536862664 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66dc0cf5a0 0x7f66dc15e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:30.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.601+0000 7f66e17da700 1 -- 172.21.15.22:0/1536862664 >> 172.21.15.22:0/1536862664 conn(0x7f66dc09c2e0 msgr2=0x7f66dc06b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:30.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.601+0000 7f66e17da700 1 -- 172.21.15.22:0/1536862664 shutdown_connections 2023-07-06T01:20:30.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:30.601+0000 7f66e17da700 1 -- 172.21.15.22:0/1536862664 wait complete. 2023-07-06T01:20:30.606 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:20:30.615 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-07-06T01:20:30.015019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219792,"kb_used_data":4176,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754928,"statfs":{"total":773094113280,"available":772869046272,"internally_reserved":0,"allocated":4276224,"data_stored":2435754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.183673"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855626+0000","last_change":"2023-07-06T01:20:24.852545+0000","last_active":"2023-07-06T01:20:24.855626+0000","last_peered":"2023-07-06T01:20:24.855626+0000","last_clean":"2023-07-06T01:20:24.855626+0000","last_became_active":"2023-07-06T01:20:24.852249+0000","last_became_peered":"2023-07-06T01:20:24.852249+0000","last_unstale":"2023-07-06T01:20:24.855626+0000","last_undegraded":"2023-07-06T01:20:24.855626+0000","last_fullsized":"2023-07-06T01:20:24.855626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:17.181181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.854187+0000","last_change":"2023-07-06T01:20:24.850725+0000","last_active":"2023-07-06T01:20:24.854187+0000","last_peered":"2023-07-06T01:20:24.854187+0000","last_clean":"2023-07-06T01:20:24.854187+0000","last_became_active":"2023-07-06T01:20:24.849359+0000","last_became_peered":"2023-07-06T01:20:24.849359+0000","last_unstale":"2023-07-06T01:20:24.854187+0000","last_undegraded":"2023-07-06T01:20:24.854187+0000","last_fullsized":"2023-07-06T01:20:24.854187+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:20.409326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834789+0000","last_change":"2023-07-06T01:20:24.855847+0000","last_active":"2023-07-06T01:20:25.834789+0000","last_peered":"2023-07-06T01:20:25.834789+0000","last_clean":"2023-07-06T01:20:25.834789+0000","last_became_active":"2023-07-06T01:20:24.855571+0000","last_became_peered":"2023-07-06T01:20:24.855571+0000","last_unstale":"2023-07-06T01:20:25.834789+0000","last_undegraded":"2023-07-06T01:20:25.834789+0000","last_fullsized":"2023-07-06T01:20:25.834789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884281+0000","last_change":"2023-07-06T01:20:24.860708+0000","last_active":"2023-07-06T01:20:25.884281+0000","last_peered":"2023-07-06T01:20:25.884281+0000","last_clean":"2023-07-06T01:20:25.884281+0000","last_became_active":"2023-07-06T01:20:24.860220+0000","last_became_peered":"2023-07-06T01:20:24.860220+0000","last_unstale":"2023-07-06T01:20:25.884281+0000","last_undegraded":"2023-07-06T01:20:25.884281+0000","last_fullsized":"2023-07-06T01:20:25.884281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:42:33.239141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884127+0000","last_change":"2023-07-06T01:20:24.850342+0000","last_active":"2023-07-06T01:20:25.884127+0000","last_peered":"2023-07-06T01:20:25.884127+0000","last_clean":"2023-07-06T01:20:25.884127+0000","last_became_active":"2023-07-06T01:20:24.850083+0000","last_became_peered":"2023-07-06T01:20:24.850083+0000","last_unstale":"2023-07-06T01:20:25.884127+0000","last_undegraded":"2023-07-06T01:20:25.884127+0000","last_fullsized":"2023-07-06T01:20:25.884127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:46:35.434349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.850816+0000","last_change":"2023-07-06T01:20:24.848000+0000","last_active":"2023-07-06T01:20:24.850816+0000","last_peered":"2023-07-06T01:20:24.850816+0000","last_clean":"2023-07-06T01:20:24.850816+0000","last_became_active":"2023-07-06T01:20:24.847814+0000","last_became_peered":"2023-07-06T01:20:24.847814+0000","last_unstale":"2023-07-06T01:20:24.850816+0000","last_undegraded":"2023-07-06T01:20:24.850816+0000","last_fullsized":"2023-07-06T01:20:24.850816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:51.766923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886546+0000","last_change":"2023-07-06T01:20:24.856730+0000","last_active":"2023-07-06T01:20:25.886546+0000","last_peered":"2023-07-06T01:20:25.886546+0000","last_clean":"2023-07-06T01:20:25.886546+0000","last_became_active":"2023-07-06T01:20:24.856460+0000","last_became_peered":"2023-07-06T01:20:24.856460+0000","last_unstale":"2023-07-06T01:20:25.886546+0000","last_undegraded":"2023-07-06T01:20:25.886546+0000","last_fullsized":"2023-07-06T01:20:25.886546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:52.466998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859689+0000","last_change":"2023-07-06T01:20:24.858297+0000","last_active":"2023-07-06T01:20:24.859689+0000","last_peered":"2023-07-06T01:20:24.859689+0000","last_clean":"2023-07-06T01:20:24.859689+0000","last_became_active":"2023-07-06T01:20:24.858033+0000","last_became_peered":"2023-07-06T01:20:24.858033+0000","last_unstale":"2023-07-06T01:20:24.859689+0000","last_undegraded":"2023-07-06T01:20:24.859689+0000","last_fullsized":"2023-07-06T01:20:24.859689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:38.837442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834859+0000","last_change":"2023-07-06T01:20:24.859281+0000","last_active":"2023-07-06T01:20:25.834859+0000","last_peered":"2023-07-06T01:20:25.834859+0000","last_clean":"2023-07-06T01:20:25.834859+0000","last_became_active":"2023-07-06T01:20:24.858982+0000","last_became_peered":"2023-07-06T01:20:24.858982+0000","last_unstale":"2023-07-06T01:20:25.834859+0000","last_undegraded":"2023-07-06T01:20:25.834859+0000","last_fullsized":"2023-07-06T01:20:25.834859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:41.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886707+0000","last_change":"2023-07-06T01:20:24.854480+0000","last_active":"2023-07-06T01:20:25.886707+0000","last_peered":"2023-07-06T01:20:25.886707+0000","last_clean":"2023-07-06T01:20:25.886707+0000","last_became_active":"2023-07-06T01:20:24.854116+0000","last_became_peered":"2023-07-06T01:20:24.854116+0000","last_unstale":"2023-07-06T01:20:25.886707+0000","last_undegraded":"2023-07-06T01:20:25.886707+0000","last_fullsized":"2023-07-06T01:20:25.886707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:12.103457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886974+0000","last_change":"2023-07-06T01:20:24.854685+0000","last_active":"2023-07-06T01:20:25.886974+0000","last_peered":"2023-07-06T01:20:25.886974+0000","last_clean":"2023-07-06T01:20:25.886974+0000","last_became_active":"2023-07-06T01:20:24.853605+0000","last_became_peered":"2023-07-06T01:20:24.853605+0000","last_unstale":"2023-07-06T01:20:25.886974+0000","last_undegraded":"2023-07-06T01:20:25.886974+0000","last_fullsized":"2023-07-06T01:20:25.886974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:54:12.127760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834625+0000","last_change":"2023-07-06T01:20:24.853728+0000","last_active":"2023-07-06T01:20:25.834625+0000","last_peered":"2023-07-06T01:20:25.834625+0000","last_clean":"2023-07-06T01:20:25.834625+0000","last_became_active":"2023-07-06T01:20:24.853427+0000","last_became_peered":"2023-07-06T01:20:24.853427+0000","last_unstale":"2023-07-06T01:20:25.834625+0000","last_undegraded":"2023-07-06T01:20:25.834625+0000","last_fullsized":"2023-07-06T01:20:25.834625+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:31.557538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886831+0000","last_change":"2023-07-06T01:20:24.856012+0000","last_active":"2023-07-06T01:20:25.886831+0000","last_peered":"2023-07-06T01:20:25.886831+0000","last_clean":"2023-07-06T01:20:25.886831+0000","last_became_active":"2023-07-06T01:20:24.855762+0000","last_became_peered":"2023-07-06T01:20:24.855762+0000","last_unstale":"2023-07-06T01:20:25.886831+0000","last_undegraded":"2023-07-06T01:20:25.886831+0000","last_fullsized":"2023-07-06T01:20:25.886831+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:45.220808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.423022+0000","last_change":"2023-07-06T01:20:24.855945+0000","last_active":"2023-07-06T01:20:26.423022+0000","last_peered":"2023-07-06T01:20:26.423022+0000","last_clean":"2023-07-06T01:20:26.423022+0000","last_became_active":"2023-07-06T01:20:24.855312+0000","last_became_peered":"2023-07-06T01:20:24.855312+0000","last_unstale":"2023-07-06T01:20:26.423022+0000","last_undegraded":"2023-07-06T01:20:26.423022+0000","last_fullsized":"2023-07-06T01:20:26.423022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:08:48.865776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886397+0000","last_change":"2023-07-06T01:20:24.857423+0000","last_active":"2023-07-06T01:20:25.886397+0000","last_peered":"2023-07-06T01:20:25.886397+0000","last_clean":"2023-07-06T01:20:25.886397+0000","last_became_active":"2023-07-06T01:20:24.857164+0000","last_became_peered":"2023-07-06T01:20:24.857164+0000","last_unstale":"2023-07-06T01:20:25.886397+0000","last_undegraded":"2023-07-06T01:20:25.886397+0000","last_fullsized":"2023-07-06T01:20:25.886397+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836435+0000","last_change":"2023-07-06T01:20:24.858965+0000","last_active":"2023-07-06T01:20:25.836435+0000","last_peered":"2023-07-06T01:20:25.836435+0000","last_clean":"2023-07-06T01:20:25.836435+0000","last_became_active":"2023-07-06T01:20:24.858671+0000","last_became_peered":"2023-07-06T01:20:24.858671+0000","last_unstale":"2023-07-06T01:20:25.836435+0000","last_undegraded":"2023-07-06T01:20:25.836435+0000","last_fullsized":"2023-07-06T01:20:25.836435+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:02:07.880193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836250+0000","last_change":"2023-07-06T01:20:24.855887+0000","last_active":"2023-07-06T01:20:25.836250+0000","last_peered":"2023-07-06T01:20:25.836250+0000","last_clean":"2023-07-06T01:20:25.836250+0000","last_became_active":"2023-07-06T01:20:24.855621+0000","last_became_peered":"2023-07-06T01:20:24.855621+0000","last_unstale":"2023-07-06T01:20:25.836250+0000","last_undegraded":"2023-07-06T01:20:25.836250+0000","last_fullsized":"2023-07-06T01:20:25.836250+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:49.817004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836700+0000","last_change":"2023-07-06T01:20:24.860237+0000","last_active":"2023-07-06T01:20:25.836700+0000","last_peered":"2023-07-06T01:20:25.836700+0000","last_clean":"2023-07-06T01:20:25.836700+0000","last_became_active":"2023-07-06T01:20:24.859324+0000","last_became_peered":"2023-07-06T01:20:24.859324+0000","last_unstale":"2023-07-06T01:20:25.836700+0000","last_undegraded":"2023-07-06T01:20:25.836700+0000","last_fullsized":"2023-07-06T01:20:25.836700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:50.731526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884196+0000","last_change":"2023-07-06T01:20:24.852638+0000","last_active":"2023-07-06T01:20:25.884196+0000","last_peered":"2023-07-06T01:20:25.884196+0000","last_clean":"2023-07-06T01:20:25.884196+0000","last_became_active":"2023-07-06T01:20:24.852341+0000","last_became_peered":"2023-07-06T01:20:24.852341+0000","last_unstale":"2023-07-06T01:20:25.884196+0000","last_undegraded":"2023-07-06T01:20:25.884196+0000","last_fullsized":"2023-07-06T01:20:25.884196+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:17:34.632929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836872+0000","last_change":"2023-07-06T01:20:24.859626+0000","last_active":"2023-07-06T01:20:25.836872+0000","last_peered":"2023-07-06T01:20:25.836872+0000","last_clean":"2023-07-06T01:20:25.836872+0000","last_became_active":"2023-07-06T01:20:24.859274+0000","last_became_peered":"2023-07-06T01:20:24.859274+0000","last_unstale":"2023-07-06T01:20:25.836872+0000","last_undegraded":"2023-07-06T01:20:25.836872+0000","last_fullsized":"2023-07-06T01:20:25.836872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:03:24.140950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886696+0000","last_change":"2023-07-06T01:20:24.854443+0000","last_active":"2023-07-06T01:20:25.886696+0000","last_peered":"2023-07-06T01:20:25.886696+0000","last_clean":"2023-07-06T01:20:25.886696+0000","last_became_active":"2023-07-06T01:20:24.853256+0000","last_became_peered":"2023-07-06T01:20:24.853256+0000","last_unstale":"2023-07-06T01:20:25.886696+0000","last_undegraded":"2023-07-06T01:20:25.886696+0000","last_fullsized":"2023-07-06T01:20:25.886696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:20.409326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422484+0000","last_change":"2023-07-06T01:20:24.849709+0000","last_active":"2023-07-06T01:20:26.422484+0000","last_peered":"2023-07-06T01:20:26.422484+0000","last_clean":"2023-07-06T01:20:26.422484+0000","last_became_active":"2023-07-06T01:20:24.849426+0000","last_became_peered":"2023-07-06T01:20:24.849426+0000","last_unstale":"2023-07-06T01:20:26.422484+0000","last_undegraded":"2023-07-06T01:20:26.422484+0000","last_fullsized":"2023-07-06T01:20:26.422484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:15:43.509777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836223+0000","last_change":"2023-07-06T01:20:24.854999+0000","last_active":"2023-07-06T01:20:25.836223+0000","last_peered":"2023-07-06T01:20:25.836223+0000","last_clean":"2023-07-06T01:20:25.836223+0000","last_became_active":"2023-07-06T01:20:24.854710+0000","last_became_peered":"2023-07-06T01:20:24.854710+0000","last_unstale":"2023-07-06T01:20:25.836223+0000","last_undegraded":"2023-07-06T01:20:25.836223+0000","last_fullsized":"2023-07-06T01:20:25.836223+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:35:50.376971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884398+0000","last_change":"2023-07-06T01:20:24.852693+0000","last_active":"2023-07-06T01:20:25.884398+0000","last_peered":"2023-07-06T01:20:25.884398+0000","last_clean":"2023-07-06T01:20:25.884398+0000","last_became_active":"2023-07-06T01:20:24.852395+0000","last_became_peered":"2023-07-06T01:20:24.852395+0000","last_unstale":"2023-07-06T01:20:25.884398+0000","last_undegraded":"2023-07-06T01:20:25.884398+0000","last_fullsized":"2023-07-06T01:20:25.884398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:35.585237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836707+0000","last_change":"2023-07-06T01:20:24.856089+0000","last_active":"2023-07-06T01:20:25.836707+0000","last_peered":"2023-07-06T01:20:25.836707+0000","last_clean":"2023-07-06T01:20:25.836707+0000","last_became_active":"2023-07-06T01:20:24.855834+0000","last_became_peered":"2023-07-06T01:20:24.855834+0000","last_unstale":"2023-07-06T01:20:25.836707+0000","last_undegraded":"2023-07-06T01:20:25.836707+0000","last_fullsized":"2023-07-06T01:20:25.836707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:03.093141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859016+0000","last_change":"2023-07-06T01:20:24.856754+0000","last_active":"2023-07-06T01:20:24.859016+0000","last_peered":"2023-07-06T01:20:24.859016+0000","last_clean":"2023-07-06T01:20:24.859016+0000","last_became_active":"2023-07-06T01:20:24.856396+0000","last_became_peered":"2023-07-06T01:20:24.856396+0000","last_unstale":"2023-07-06T01:20:24.859016+0000","last_undegraded":"2023-07-06T01:20:24.859016+0000","last_fullsized":"2023-07-06T01:20:24.859016+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:06:02.668449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835952+0000","last_change":"2023-07-06T01:20:24.854996+0000","last_active":"2023-07-06T01:20:25.835952+0000","last_peered":"2023-07-06T01:20:25.835952+0000","last_clean":"2023-07-06T01:20:25.835952+0000","last_became_active":"2023-07-06T01:20:24.854737+0000","last_became_peered":"2023-07-06T01:20:24.854737+0000","last_unstale":"2023-07-06T01:20:25.835952+0000","last_undegraded":"2023-07-06T01:20:25.835952+0000","last_fullsized":"2023-07-06T01:20:25.835952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:51.766923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886641+0000","last_change":"2023-07-06T01:20:24.856804+0000","last_active":"2023-07-06T01:20:25.886641+0000","last_peered":"2023-07-06T01:20:25.886641+0000","last_clean":"2023-07-06T01:20:25.886641+0000","last_became_active":"2023-07-06T01:20:24.856490+0000","last_became_peered":"2023-07-06T01:20:24.856490+0000","last_unstale":"2023-07-06T01:20:25.886641+0000","last_undegraded":"2023-07-06T01:20:25.886641+0000","last_fullsized":"2023-07-06T01:20:25.886641+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:23:07.815642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835630+0000","last_change":"2023-07-06T01:20:24.850799+0000","last_active":"2023-07-06T01:20:25.835630+0000","last_peered":"2023-07-06T01:20:25.835630+0000","last_clean":"2023-07-06T01:20:25.835630+0000","last_became_active":"2023-07-06T01:20:24.850463+0000","last_became_peered":"2023-07-06T01:20:24.850463+0000","last_unstale":"2023-07-06T01:20:25.835630+0000","last_undegraded":"2023-07-06T01:20:25.835630+0000","last_fullsized":"2023-07-06T01:20:25.835630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:50.421447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835944+0000","last_change":"2023-07-06T01:20:24.857969+0000","last_active":"2023-07-06T01:20:25.835944+0000","last_peered":"2023-07-06T01:20:25.835944+0000","last_clean":"2023-07-06T01:20:25.835944+0000","last_became_active":"2023-07-06T01:20:24.857674+0000","last_became_peered":"2023-07-06T01:20:24.857674+0000","last_unstale":"2023-07-06T01:20:25.835944+0000","last_undegraded":"2023-07-06T01:20:25.835944+0000","last_fullsized":"2023-07-06T01:20:25.835944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:57.090630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835732+0000","last_change":"2023-07-06T01:20:24.850856+0000","last_active":"2023-07-06T01:20:25.835732+0000","last_peered":"2023-07-06T01:20:25.835732+0000","last_clean":"2023-07-06T01:20:25.835732+0000","last_became_active":"2023-07-06T01:20:24.850484+0000","last_became_peered":"2023-07-06T01:20:24.850484+0000","last_unstale":"2023-07-06T01:20:25.835732+0000","last_undegraded":"2023-07-06T01:20:25.835732+0000","last_fullsized":"2023-07-06T01:20:25.835732+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:38:28.819319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.856018+0000","last_change":"2023-07-06T01:20:24.854034+0000","last_active":"2023-07-06T01:20:24.856018+0000","last_peered":"2023-07-06T01:20:24.856018+0000","last_clean":"2023-07-06T01:20:24.856018+0000","last_became_active":"2023-07-06T01:20:24.853820+0000","last_became_peered":"2023-07-06T01:20:24.853820+0000","last_unstale":"2023-07-06T01:20:24.856018+0000","last_undegraded":"2023-07-06T01:20:24.856018+0000","last_fullsized":"2023-07-06T01:20:24.856018+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:57.090630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836081+0000","last_change":"2023-07-06T01:20:24.857755+0000","last_active":"2023-07-06T01:20:25.836081+0000","last_peered":"2023-07-06T01:20:25.836081+0000","last_clean":"2023-07-06T01:20:25.836081+0000","last_became_active":"2023-07-06T01:20:24.855354+0000","last_became_peered":"2023-07-06T01:20:24.855354+0000","last_unstale":"2023-07-06T01:20:25.836081+0000","last_undegraded":"2023-07-06T01:20:25.836081+0000","last_fullsized":"2023-07-06T01:20:25.836081+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:55:10.623339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836884+0000","last_change":"2023-07-06T01:20:23.849203+0000","last_active":"2023-07-06T01:20:25.836884+0000","last_peered":"2023-07-06T01:20:25.836884+0000","last_clean":"2023-07-06T01:20:25.836884+0000","last_became_active":"2023-07-06T01:20:23.848883+0000","last_became_peered":"2023-07-06T01:20:23.848883+0000","last_unstale":"2023-07-06T01:20:25.836884+0000","last_undegraded":"2023-07-06T01:20:25.836884+0000","last_fullsized":"2023-07-06T01:20:25.836884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:06.770007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.860470+0000","last_change":"2023-07-06T01:20:24.859359+0000","last_active":"2023-07-06T01:20:24.860470+0000","last_peered":"2023-07-06T01:20:24.860470+0000","last_clean":"2023-07-06T01:20:24.860470+0000","last_became_active":"2023-07-06T01:20:24.859152+0000","last_became_peered":"2023-07-06T01:20:24.859152+0000","last_unstale":"2023-07-06T01:20:24.860470+0000","last_undegraded":"2023-07-06T01:20:24.860470+0000","last_fullsized":"2023-07-06T01:20:24.860470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:01.181469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834689+0000","last_change":"2023-07-06T01:20:23.845980+0000","last_active":"2023-07-06T01:20:25.834689+0000","last_peered":"2023-07-06T01:20:25.834689+0000","last_clean":"2023-07-06T01:20:25.834689+0000","last_became_active":"2023-07-06T01:20:23.845696+0000","last_became_peered":"2023-07-06T01:20:23.845696+0000","last_unstale":"2023-07-06T01:20:25.834689+0000","last_undegraded":"2023-07-06T01:20:25.834689+0000","last_fullsized":"2023-07-06T01:20:25.834689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:58.579906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422869+0000","last_change":"2023-07-06T01:20:24.851207+0000","last_active":"2023-07-06T01:20:26.422869+0000","last_peered":"2023-07-06T01:20:26.422869+0000","last_clean":"2023-07-06T01:20:26.422869+0000","last_became_active":"2023-07-06T01:20:24.850929+0000","last_became_peered":"2023-07-06T01:20:24.850929+0000","last_unstale":"2023-07-06T01:20:26.422869+0000","last_undegraded":"2023-07-06T01:20:26.422869+0000","last_fullsized":"2023-07-06T01:20:26.422869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:38:16.612537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.858947+0000","last_change":"2023-07-06T01:20:23.846742+0000","last_active":"2023-07-06T01:20:24.858947+0000","last_peered":"2023-07-06T01:20:24.858947+0000","last_clean":"2023-07-06T01:20:24.858947+0000","last_became_active":"2023-07-06T01:20:23.846440+0000","last_became_peered":"2023-07-06T01:20:23.846440+0000","last_unstale":"2023-07-06T01:20:24.858947+0000","last_undegraded":"2023-07-06T01:20:24.858947+0000","last_fullsized":"2023-07-06T01:20:24.858947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:06.770007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862194+0000","last_change":"2023-07-06T01:20:24.861558+0000","last_active":"2023-07-06T01:20:24.862194+0000","last_peered":"2023-07-06T01:20:24.862194+0000","last_clean":"2023-07-06T01:20:24.862194+0000","last_became_active":"2023-07-06T01:20:24.860857+0000","last_became_peered":"2023-07-06T01:20:24.860857+0000","last_unstale":"2023-07-06T01:20:24.862194+0000","last_undegraded":"2023-07-06T01:20:24.862194+0000","last_fullsized":"2023-07-06T01:20:24.862194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:52.466998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422998+0000","last_change":"2023-07-06T01:20:23.848100+0000","last_active":"2023-07-06T01:20:26.422998+0000","last_peered":"2023-07-06T01:20:26.422998+0000","last_clean":"2023-07-06T01:20:26.422998+0000","last_became_active":"2023-07-06T01:20:23.847541+0000","last_became_peered":"2023-07-06T01:20:23.847541+0000","last_unstale":"2023-07-06T01:20:26.422998+0000","last_undegraded":"2023-07-06T01:20:26.422998+0000","last_fullsized":"2023-07-06T01:20:26.422998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:35:05.552930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834373+0000","last_change":"2023-07-06T01:20:24.839527+0000","last_active":"2023-07-06T01:20:25.834373+0000","last_peered":"2023-07-06T01:20:25.834373+0000","last_clean":"2023-07-06T01:20:25.834373+0000","last_became_active":"2023-07-06T01:20:24.839201+0000","last_became_peered":"2023-07-06T01:20:24.839201+0000","last_unstale":"2023-07-06T01:20:25.834373+0000","last_undegraded":"2023-07-06T01:20:25.834373+0000","last_fullsized":"2023-07-06T01:20:25.834373+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:57:20.290658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886526+0000","last_change":"2023-07-06T01:20:23.853509+0000","last_active":"2023-07-06T01:20:25.886526+0000","last_peered":"2023-07-06T01:20:25.886526+0000","last_clean":"2023-07-06T01:20:25.886526+0000","last_became_active":"2023-07-06T01:20:23.853060+0000","last_became_peered":"2023-07-06T01:20:23.853060+0000","last_unstale":"2023-07-06T01:20:25.886526+0000","last_undegraded":"2023-07-06T01:20:25.886526+0000","last_fullsized":"2023-07-06T01:20:25.886526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:40.214108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886380+0000","last_change":"2023-07-06T01:20:24.856240+0000","last_active":"2023-07-06T01:20:25.886380+0000","last_peered":"2023-07-06T01:20:25.886380+0000","last_clean":"2023-07-06T01:20:25.886380+0000","last_became_active":"2023-07-06T01:20:24.853426+0000","last_became_peered":"2023-07-06T01:20:24.853426+0000","last_unstale":"2023-07-06T01:20:25.886380+0000","last_undegraded":"2023-07-06T01:20:25.886380+0000","last_fullsized":"2023-07-06T01:20:25.886380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:59.584568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836264+0000","last_change":"2023-07-06T01:20:23.853484+0000","last_active":"2023-07-06T01:20:25.836264+0000","last_peered":"2023-07-06T01:20:25.836264+0000","last_clean":"2023-07-06T01:20:25.836264+0000","last_became_active":"2023-07-06T01:20:23.853173+0000","last_became_peered":"2023-07-06T01:20:23.853173+0000","last_unstale":"2023-07-06T01:20:25.836264+0000","last_undegraded":"2023-07-06T01:20:25.836264+0000","last_fullsized":"2023-07-06T01:20:25.836264+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:55:15.600714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422525+0000","last_change":"2023-07-06T01:20:24.856110+0000","last_active":"2023-07-06T01:20:26.422525+0000","last_peered":"2023-07-06T01:20:26.422525+0000","last_clean":"2023-07-06T01:20:26.422525+0000","last_became_active":"2023-07-06T01:20:24.855686+0000","last_became_peered":"2023-07-06T01:20:24.855686+0000","last_unstale":"2023-07-06T01:20:26.422525+0000","last_undegraded":"2023-07-06T01:20:26.422525+0000","last_fullsized":"2023-07-06T01:20:26.422525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:04:28.309392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836572+0000","last_change":"2023-07-06T01:20:23.849040+0000","last_active":"2023-07-06T01:20:24.836572+0000","last_peered":"2023-07-06T01:20:24.836572+0000","last_clean":"2023-07-06T01:20:24.836572+0000","last_became_active":"2023-07-06T01:20:23.848756+0000","last_became_peered":"2023-07-06T01:20:23.848756+0000","last_unstale":"2023-07-06T01:20:24.836572+0000","last_undegraded":"2023-07-06T01:20:24.836572+0000","last_fullsized":"2023-07-06T01:20:24.836572+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:05:00.225120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855846+0000","last_change":"2023-07-06T01:20:24.849751+0000","last_active":"2023-07-06T01:20:24.855846+0000","last_peered":"2023-07-06T01:20:24.855846+0000","last_clean":"2023-07-06T01:20:24.855846+0000","last_became_active":"2023-07-06T01:20:24.849475+0000","last_became_peered":"2023-07-06T01:20:24.849475+0000","last_unstale":"2023-07-06T01:20:24.855846+0000","last_undegraded":"2023-07-06T01:20:24.855846+0000","last_fullsized":"2023-07-06T01:20:24.855846+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422290+0000","last_change":"2023-07-06T01:20:23.842790+0000","last_active":"2023-07-06T01:20:26.422290+0000","last_peered":"2023-07-06T01:20:26.422290+0000","last_clean":"2023-07-06T01:20:26.422290+0000","last_became_active":"2023-07-06T01:20:23.842488+0000","last_became_peered":"2023-07-06T01:20:23.842488+0000","last_unstale":"2023-07-06T01:20:26.422290+0000","last_undegraded":"2023-07-06T01:20:26.422290+0000","last_fullsized":"2023-07-06T01:20:26.422290+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:17:05.908517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884424+0000","last_change":"2023-07-06T01:20:24.860540+0000","last_active":"2023-07-06T01:20:25.884424+0000","last_peered":"2023-07-06T01:20:25.884424+0000","last_clean":"2023-07-06T01:20:25.884424+0000","last_became_active":"2023-07-06T01:20:24.859869+0000","last_became_peered":"2023-07-06T01:20:24.859869+0000","last_unstale":"2023-07-06T01:20:25.884424+0000","last_undegraded":"2023-07-06T01:20:25.884424+0000","last_fullsized":"2023-07-06T01:20:25.884424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:23.281099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.845141+0000","last_change":"2023-07-06T01:20:23.848957+0000","last_active":"2023-07-06T01:20:24.845141+0000","last_peered":"2023-07-06T01:20:24.845141+0000","last_clean":"2023-07-06T01:20:24.845141+0000","last_became_active":"2023-07-06T01:20:23.848646+0000","last_became_peered":"2023-07-06T01:20:23.848646+0000","last_unstale":"2023-07-06T01:20:24.845141+0000","last_undegraded":"2023-07-06T01:20:24.845141+0000","last_fullsized":"2023-07-06T01:20:24.845141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:35:50.683696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836461+0000","last_change":"2023-07-06T01:20:24.853193+0000","last_active":"2023-07-06T01:20:25.836461+0000","last_peered":"2023-07-06T01:20:25.836461+0000","last_clean":"2023-07-06T01:20:25.836461+0000","last_became_active":"2023-07-06T01:20:24.852778+0000","last_became_peered":"2023-07-06T01:20:24.852778+0000","last_unstale":"2023-07-06T01:20:25.836461+0000","last_undegraded":"2023-07-06T01:20:25.836461+0000","last_fullsized":"2023-07-06T01:20:25.836461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:30:29.768004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886834+0000","last_change":"2023-07-06T01:20:23.847858+0000","last_active":"2023-07-06T01:20:25.886834+0000","last_peered":"2023-07-06T01:20:25.886834+0000","last_clean":"2023-07-06T01:20:25.886834+0000","last_became_active":"2023-07-06T01:20:23.847549+0000","last_became_peered":"2023-07-06T01:20:23.847549+0000","last_unstale":"2023-07-06T01:20:25.886834+0000","last_undegraded":"2023-07-06T01:20:25.886834+0000","last_fullsized":"2023-07-06T01:20:25.886834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:53:56.761538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834565+0000","last_change":"2023-07-06T01:20:24.848275+0000","last_active":"2023-07-06T01:20:25.834565+0000","last_peered":"2023-07-06T01:20:25.834565+0000","last_clean":"2023-07-06T01:20:25.834565+0000","last_became_active":"2023-07-06T01:20:24.847949+0000","last_became_peered":"2023-07-06T01:20:24.847949+0000","last_unstale":"2023-07-06T01:20:25.834565+0000","last_undegraded":"2023-07-06T01:20:25.834565+0000","last_fullsized":"2023-07-06T01:20:25.834565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:24:03.775615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887068+0000","last_change":"2023-07-06T01:20:23.849878+0000","last_active":"2023-07-06T01:20:25.887068+0000","last_peered":"2023-07-06T01:20:25.887068+0000","last_clean":"2023-07-06T01:20:25.887068+0000","last_became_active":"2023-07-06T01:20:23.849573+0000","last_became_peered":"2023-07-06T01:20:23.849573+0000","last_unstale":"2023-07-06T01:20:25.887068+0000","last_undegraded":"2023-07-06T01:20:25.887068+0000","last_fullsized":"2023-07-06T01:20:25.887068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:12:26.209870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422768+0000","last_change":"2023-07-06T01:20:24.854277+0000","last_active":"2023-07-06T01:20:26.422768+0000","last_peered":"2023-07-06T01:20:26.422768+0000","last_clean":"2023-07-06T01:20:26.422768+0000","last_became_active":"2023-07-06T01:20:24.853981+0000","last_became_peered":"2023-07-06T01:20:24.853981+0000","last_unstale":"2023-07-06T01:20:26.422768+0000","last_undegraded":"2023-07-06T01:20:26.422768+0000","last_fullsized":"2023-07-06T01:20:26.422768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:59:32.105014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.868009+0000","last_change":"2023-07-06T01:20:23.845648+0000","last_active":"2023-07-06T01:20:24.868009+0000","last_peered":"2023-07-06T01:20:24.868009+0000","last_clean":"2023-07-06T01:20:24.868009+0000","last_became_active":"2023-07-06T01:20:23.845340+0000","last_became_peered":"2023-07-06T01:20:23.845340+0000","last_unstale":"2023-07-06T01:20:24.868009+0000","last_undegraded":"2023-07-06T01:20:24.868009+0000","last_fullsized":"2023-07-06T01:20:24.868009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:39:35.605373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834521+0000","last_change":"2023-07-06T01:20:24.858282+0000","last_active":"2023-07-06T01:20:25.834521+0000","last_peered":"2023-07-06T01:20:25.834521+0000","last_clean":"2023-07-06T01:20:25.834521+0000","last_became_active":"2023-07-06T01:20:24.857879+0000","last_became_peered":"2023-07-06T01:20:24.857879+0000","last_unstale":"2023-07-06T01:20:25.834521+0000","last_undegraded":"2023-07-06T01:20:25.834521+0000","last_fullsized":"2023-07-06T01:20:25.834521+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:22:46.902965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855788+0000","last_change":"2023-07-06T01:20:23.841745+0000","last_active":"2023-07-06T01:20:24.855788+0000","last_peered":"2023-07-06T01:20:24.855788+0000","last_clean":"2023-07-06T01:20:24.855788+0000","last_became_active":"2023-07-06T01:20:23.841438+0000","last_became_peered":"2023-07-06T01:20:23.841438+0000","last_unstale":"2023-07-06T01:20:24.855788+0000","last_undegraded":"2023-07-06T01:20:24.855788+0000","last_fullsized":"2023-07-06T01:20:24.855788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:54:11.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855590+0000","last_change":"2023-07-06T01:20:24.854272+0000","last_active":"2023-07-06T01:20:24.855590+0000","last_peered":"2023-07-06T01:20:24.855590+0000","last_clean":"2023-07-06T01:20:24.855590+0000","last_became_active":"2023-07-06T01:20:24.854002+0000","last_became_peered":"2023-07-06T01:20:24.854002+0000","last_unstale":"2023-07-06T01:20:24.855590+0000","last_undegraded":"2023-07-06T01:20:24.855590+0000","last_fullsized":"2023-07-06T01:20:24.855590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:23:07.815642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836795+0000","last_change":"2023-07-06T01:20:23.854212+0000","last_active":"2023-07-06T01:20:24.836795+0000","last_peered":"2023-07-06T01:20:24.836795+0000","last_clean":"2023-07-06T01:20:24.836795+0000","last_became_active":"2023-07-06T01:20:23.853928+0000","last_became_peered":"2023-07-06T01:20:23.853928+0000","last_unstale":"2023-07-06T01:20:24.836795+0000","last_undegraded":"2023-07-06T01:20:24.836795+0000","last_fullsized":"2023-07-06T01:20:24.836795+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:49.726864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835989+0000","last_change":"2023-07-06T01:20:24.855007+0000","last_active":"2023-07-06T01:20:25.835989+0000","last_peered":"2023-07-06T01:20:25.835989+0000","last_clean":"2023-07-06T01:20:25.835989+0000","last_became_active":"2023-07-06T01:20:24.854610+0000","last_became_peered":"2023-07-06T01:20:24.854610+0000","last_unstale":"2023-07-06T01:20:25.835989+0000","last_undegraded":"2023-07-06T01:20:25.835989+0000","last_fullsized":"2023-07-06T01:20:25.835989+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:27:38.804964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835969+0000","last_change":"2023-07-06T01:20:23.847446+0000","last_active":"2023-07-06T01:20:25.835969+0000","last_peered":"2023-07-06T01:20:25.835969+0000","last_clean":"2023-07-06T01:20:25.835969+0000","last_became_active":"2023-07-06T01:20:23.846929+0000","last_became_peered":"2023-07-06T01:20:23.846929+0000","last_unstale":"2023-07-06T01:20:25.835969+0000","last_undegraded":"2023-07-06T01:20:25.835969+0000","last_fullsized":"2023-07-06T01:20:25.835969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:05:00.225120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855866+0000","last_change":"2023-07-06T01:20:24.855191+0000","last_active":"2023-07-06T01:20:24.855866+0000","last_peered":"2023-07-06T01:20:24.855866+0000","last_clean":"2023-07-06T01:20:24.855866+0000","last_became_active":"2023-07-06T01:20:24.854898+0000","last_became_peered":"2023-07-06T01:20:24.854898+0000","last_unstale":"2023-07-06T01:20:24.855866+0000","last_undegraded":"2023-07-06T01:20:24.855866+0000","last_fullsized":"2023-07-06T01:20:24.855866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.419832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'8","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836552+0000","last_change":"2023-07-06T01:20:23.853463+0000","last_active":"2023-07-06T01:20:25.836552+0000","last_peered":"2023-07-06T01:20:25.836552+0000","last_clean":"2023-07-06T01:20:25.836552+0000","last_became_active":"2023-07-06T01:20:23.853104+0000","last_became_peered":"2023-07-06T01:20:23.853104+0000","last_unstale":"2023-07-06T01:20:25.836552+0000","last_undegraded":"2023-07-06T01:20:25.836552+0000","last_fullsized":"2023-07-06T01:20:25.836552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:59:04.523315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862166+0000","last_change":"2023-07-06T01:20:24.861072+0000","last_active":"2023-07-06T01:20:24.862166+0000","last_peered":"2023-07-06T01:20:24.862166+0000","last_clean":"2023-07-06T01:20:24.862166+0000","last_became_active":"2023-07-06T01:20:24.860745+0000","last_became_peered":"2023-07-06T01:20:24.860745+0000","last_unstale":"2023-07-06T01:20:24.862166+0000","last_undegraded":"2023-07-06T01:20:24.862166+0000","last_fullsized":"2023-07-06T01:20:24.862166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:45.220808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887003+0000","last_change":"2023-07-06T01:20:23.848801+0000","last_active":"2023-07-06T01:20:25.887003+0000","last_peered":"2023-07-06T01:20:25.887003+0000","last_clean":"2023-07-06T01:20:25.887003+0000","last_became_active":"2023-07-06T01:20:23.848412+0000","last_became_peered":"2023-07-06T01:20:23.848412+0000","last_unstale":"2023-07-06T01:20:25.887003+0000","last_undegraded":"2023-07-06T01:20:25.887003+0000","last_fullsized":"2023-07-06T01:20:25.887003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:35:50.683696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886952+0000","last_change":"2023-07-06T01:20:24.854853+0000","last_active":"2023-07-06T01:20:25.886952+0000","last_peered":"2023-07-06T01:20:25.886952+0000","last_clean":"2023-07-06T01:20:25.886952+0000","last_became_active":"2023-07-06T01:20:24.854159+0000","last_became_peered":"2023-07-06T01:20:24.854159+0000","last_unstale":"2023-07-06T01:20:25.886952+0000","last_undegraded":"2023-07-06T01:20:25.886952+0000","last_fullsized":"2023-07-06T01:20:25.886952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.419832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422967+0000","last_change":"2023-07-06T01:20:23.850450+0000","last_active":"2023-07-06T01:20:26.422967+0000","last_peered":"2023-07-06T01:20:26.422967+0000","last_clean":"2023-07-06T01:20:26.422967+0000","last_became_active":"2023-07-06T01:20:23.850157+0000","last_became_peered":"2023-07-06T01:20:23.850157+0000","last_unstale":"2023-07-06T01:20:26.422967+0000","last_undegraded":"2023-07-06T01:20:26.422967+0000","last_fullsized":"2023-07-06T01:20:26.422967+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:03:07.971132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422880+0000","last_change":"2023-07-06T01:20:24.843242+0000","last_active":"2023-07-06T01:20:26.422880+0000","last_peered":"2023-07-06T01:20:26.422880+0000","last_clean":"2023-07-06T01:20:26.422880+0000","last_became_active":"2023-07-06T01:20:24.842910+0000","last_became_peered":"2023-07-06T01:20:24.842910+0000","last_unstale":"2023-07-06T01:20:26.422880+0000","last_undegraded":"2023-07-06T01:20:26.422880+0000","last_fullsized":"2023-07-06T01:20:26.422880+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:11:05.585857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884278+0000","last_change":"2023-07-06T01:20:23.846883+0000","last_active":"2023-07-06T01:20:25.884278+0000","last_peered":"2023-07-06T01:20:25.884278+0000","last_clean":"2023-07-06T01:20:25.884278+0000","last_became_active":"2023-07-06T01:20:23.846566+0000","last_became_peered":"2023-07-06T01:20:23.846566+0000","last_unstale":"2023-07-06T01:20:25.884278+0000","last_undegraded":"2023-07-06T01:20:25.884278+0000","last_fullsized":"2023-07-06T01:20:25.884278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:44:49.870273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884313+0000","last_change":"2023-07-06T01:20:24.853783+0000","last_active":"2023-07-06T01:20:25.884313+0000","last_peered":"2023-07-06T01:20:25.884313+0000","last_clean":"2023-07-06T01:20:25.884313+0000","last_became_active":"2023-07-06T01:20:24.853427+0000","last_became_peered":"2023-07-06T01:20:24.853427+0000","last_unstale":"2023-07-06T01:20:25.884313+0000","last_undegraded":"2023-07-06T01:20:25.884313+0000","last_fullsized":"2023-07-06T01:20:25.884313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:33:25.311596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.837321+0000","last_change":"2023-07-06T01:20:23.847016+0000","last_active":"2023-07-06T01:20:24.837321+0000","last_peered":"2023-07-06T01:20:24.837321+0000","last_clean":"2023-07-06T01:20:24.837321+0000","last_became_active":"2023-07-06T01:20:23.846748+0000","last_became_peered":"2023-07-06T01:20:23.846748+0000","last_unstale":"2023-07-06T01:20:24.837321+0000","last_undegraded":"2023-07-06T01:20:24.837321+0000","last_fullsized":"2023-07-06T01:20:24.837321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:13.424430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836548+0000","last_change":"2023-07-06T01:20:24.855376+0000","last_active":"2023-07-06T01:20:25.836548+0000","last_peered":"2023-07-06T01:20:25.836548+0000","last_clean":"2023-07-06T01:20:25.836548+0000","last_became_active":"2023-07-06T01:20:24.855107+0000","last_became_peered":"2023-07-06T01:20:24.855107+0000","last_unstale":"2023-07-06T01:20:25.836548+0000","last_undegraded":"2023-07-06T01:20:25.836548+0000","last_fullsized":"2023-07-06T01:20:25.836548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:42:46.861502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884268+0000","last_change":"2023-07-06T01:20:23.847882+0000","last_active":"2023-07-06T01:20:25.884268+0000","last_peered":"2023-07-06T01:20:25.884268+0000","last_clean":"2023-07-06T01:20:25.884268+0000","last_became_active":"2023-07-06T01:20:23.847534+0000","last_became_peered":"2023-07-06T01:20:23.847534+0000","last_unstale":"2023-07-06T01:20:25.884268+0000","last_undegraded":"2023-07-06T01:20:25.884268+0000","last_fullsized":"2023-07-06T01:20:25.884268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:22:12.321678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834687+0000","last_change":"2023-07-06T01:20:24.848033+0000","last_active":"2023-07-06T01:20:25.834687+0000","last_peered":"2023-07-06T01:20:25.834687+0000","last_clean":"2023-07-06T01:20:25.834687+0000","last_became_active":"2023-07-06T01:20:24.847514+0000","last_became_peered":"2023-07-06T01:20:24.847514+0000","last_unstale":"2023-07-06T01:20:25.834687+0000","last_undegraded":"2023-07-06T01:20:25.834687+0000","last_fullsized":"2023-07-06T01:20:25.834687+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:02.805903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884363+0000","last_change":"2023-07-06T01:20:23.850960+0000","last_active":"2023-07-06T01:20:25.884363+0000","last_peered":"2023-07-06T01:20:25.884363+0000","last_clean":"2023-07-06T01:20:25.884363+0000","last_became_active":"2023-07-06T01:20:23.850699+0000","last_became_peered":"2023-07-06T01:20:23.850699+0000","last_unstale":"2023-07-06T01:20:25.884363+0000","last_undegraded":"2023-07-06T01:20:25.884363+0000","last_fullsized":"2023-07-06T01:20:25.884363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:30:39.863076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886519+0000","last_change":"2023-07-06T01:20:24.861071+0000","last_active":"2023-07-06T01:20:25.886519+0000","last_peered":"2023-07-06T01:20:25.886519+0000","last_clean":"2023-07-06T01:20:25.886519+0000","last_became_active":"2023-07-06T01:20:24.860803+0000","last_became_peered":"2023-07-06T01:20:24.860803+0000","last_unstale":"2023-07-06T01:20:25.886519+0000","last_undegraded":"2023-07-06T01:20:25.886519+0000","last_fullsized":"2023-07-06T01:20:25.886519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:17.181181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836165+0000","last_change":"2023-07-06T01:20:23.851273+0000","last_active":"2023-07-06T01:20:25.836165+0000","last_peered":"2023-07-06T01:20:25.836165+0000","last_clean":"2023-07-06T01:20:25.836165+0000","last_became_active":"2023-07-06T01:20:23.850942+0000","last_became_peered":"2023-07-06T01:20:23.850942+0000","last_unstale":"2023-07-06T01:20:25.836165+0000","last_undegraded":"2023-07-06T01:20:25.836165+0000","last_fullsized":"2023-07-06T01:20:25.836165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:01:23.333657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862183+0000","last_change":"2023-07-06T01:20:24.861226+0000","last_active":"2023-07-06T01:20:24.862183+0000","last_peered":"2023-07-06T01:20:24.862183+0000","last_clean":"2023-07-06T01:20:24.862183+0000","last_became_active":"2023-07-06T01:20:24.860944+0000","last_became_peered":"2023-07-06T01:20:24.860944+0000","last_unstale":"2023-07-06T01:20:24.862183+0000","last_undegraded":"2023-07-06T01:20:24.862183+0000","last_fullsized":"2023-07-06T01:20:24.862183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:12.103457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884125+0000","last_change":"2023-07-06T01:20:23.849860+0000","last_active":"2023-07-06T01:20:25.884125+0000","last_peered":"2023-07-06T01:20:25.884125+0000","last_clean":"2023-07-06T01:20:25.884125+0000","last_became_active":"2023-07-06T01:20:23.849507+0000","last_became_peered":"2023-07-06T01:20:23.849507+0000","last_unstale":"2023-07-06T01:20:25.884125+0000","last_undegraded":"2023-07-06T01:20:25.884125+0000","last_fullsized":"2023-07-06T01:20:25.884125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:51.413216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.423002+0000","last_change":"2023-07-06T01:20:24.853501+0000","last_active":"2023-07-06T01:20:26.423002+0000","last_peered":"2023-07-06T01:20:26.423002+0000","last_clean":"2023-07-06T01:20:26.423002+0000","last_became_active":"2023-07-06T01:20:24.853232+0000","last_became_peered":"2023-07-06T01:20:24.853232+0000","last_unstale":"2023-07-06T01:20:26.423002+0000","last_undegraded":"2023-07-06T01:20:26.423002+0000","last_fullsized":"2023-07-06T01:20:26.423002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:14:59.968797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422341+0000","last_change":"2023-07-06T01:20:23.844519+0000","last_active":"2023-07-06T01:20:26.422341+0000","last_peered":"2023-07-06T01:20:26.422341+0000","last_clean":"2023-07-06T01:20:26.422341+0000","last_became_active":"2023-07-06T01:20:23.844260+0000","last_became_peered":"2023-07-06T01:20:23.844260+0000","last_unstale":"2023-07-06T01:20:26.422341+0000","last_undegraded":"2023-07-06T01:20:26.422341+0000","last_fullsized":"2023-07-06T01:20:26.422341+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:17.854150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422622+0000","last_change":"2023-07-06T01:19:20.074195+0000","last_active":"2023-07-06T01:20:26.422622+0000","last_peered":"2023-07-06T01:20:26.422622+0000","last_clean":"2023-07-06T01:20:26.422622+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:20:26.422622+0000","last_undegraded":"2023-07-06T01:20:26.422622+0000","last_fullsized":"2023-07-06T01:20:26.422622+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:47.542825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835909+0000","last_change":"2023-07-06T01:20:24.856069+0000","last_active":"2023-07-06T01:20:25.835909+0000","last_peered":"2023-07-06T01:20:25.835909+0000","last_clean":"2023-07-06T01:20:25.835909+0000","last_became_active":"2023-07-06T01:20:24.855766+0000","last_became_peered":"2023-07-06T01:20:24.855766+0000","last_unstale":"2023-07-06T01:20:25.835909+0000","last_undegraded":"2023-07-06T01:20:25.835909+0000","last_fullsized":"2023-07-06T01:20:25.835909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:55.581029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.842093+0000","last_change":"2023-07-06T01:20:23.849570+0000","last_active":"2023-07-06T01:20:24.842093+0000","last_peered":"2023-07-06T01:20:24.842093+0000","last_clean":"2023-07-06T01:20:24.842093+0000","last_became_active":"2023-07-06T01:20:23.849255+0000","last_became_peered":"2023-07-06T01:20:23.849255+0000","last_unstale":"2023-07-06T01:20:24.842093+0000","last_undegraded":"2023-07-06T01:20:24.842093+0000","last_fullsized":"2023-07-06T01:20:24.842093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:53:56.761538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.846124+0000","last_change":"2023-07-06T01:19:19.587110+0000","last_active":"2023-07-06T01:20:24.846124+0000","last_peered":"2023-07-06T01:20:24.846124+0000","last_clean":"2023-07-06T01:20:24.846124+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:20:24.846124+0000","last_undegraded":"2023-07-06T01:20:24.846124+0000","last_fullsized":"2023-07-06T01:20:24.846124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:49:50.820241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859472+0000","last_change":"2023-07-06T01:20:24.858252+0000","last_active":"2023-07-06T01:20:24.859472+0000","last_peered":"2023-07-06T01:20:24.859472+0000","last_clean":"2023-07-06T01:20:24.859472+0000","last_became_active":"2023-07-06T01:20:24.856691+0000","last_became_peered":"2023-07-06T01:20:24.856691+0000","last_unstale":"2023-07-06T01:20:24.859472+0000","last_undegraded":"2023-07-06T01:20:24.859472+0000","last_fullsized":"2023-07-06T01:20:24.859472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:03.093141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887051+0000","last_change":"2023-07-06T01:20:23.847756+0000","last_active":"2023-07-06T01:20:25.887051+0000","last_peered":"2023-07-06T01:20:25.887051+0000","last_clean":"2023-07-06T01:20:25.887051+0000","last_became_active":"2023-07-06T01:20:23.847392+0000","last_became_peered":"2023-07-06T01:20:23.847392+0000","last_unstale":"2023-07-06T01:20:25.887051+0000","last_undegraded":"2023-07-06T01:20:25.887051+0000","last_fullsized":"2023-07-06T01:20:25.887051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:50:55.073152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":35,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836206+0000","last_change":"2023-07-06T01:19:19.584242+0000","last_active":"2023-07-06T01:20:24.836206+0000","last_peered":"2023-07-06T01:20:24.836206+0000","last_clean":"2023-07-06T01:20:24.836206+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:20:24.836206+0000","last_undegraded":"2023-07-06T01:20:24.836206+0000","last_fullsized":"2023-07-06T01:20:24.836206+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:02:18.883377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":189,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836706+0000","last_change":"2023-07-06T01:19:19.586495+0000","last_active":"2023-07-06T01:20:25.836706+0000","last_peered":"2023-07-06T01:20:25.836706+0000","last_clean":"2023-07-06T01:20:25.836706+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:20:25.836706+0000","last_undegraded":"2023-07-06T01:20:25.836706+0000","last_fullsized":"2023-07-06T01:20:25.836706+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:13:17.664579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836620+0000","last_change":"2023-07-06T01:20:24.847523+0000","last_active":"2023-07-06T01:20:25.836620+0000","last_peered":"2023-07-06T01:20:25.836620+0000","last_clean":"2023-07-06T01:20:25.836620+0000","last_became_active":"2023-07-06T01:20:24.847256+0000","last_became_peered":"2023-07-06T01:20:24.847256+0000","last_unstale":"2023-07-06T01:20:25.836620+0000","last_undegraded":"2023-07-06T01:20:25.836620+0000","last_fullsized":"2023-07-06T01:20:25.836620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:26:53.631677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.885927+0000","last_change":"2023-07-06T01:20:23.854385+0000","last_active":"2023-07-06T01:20:25.885927+0000","last_peered":"2023-07-06T01:20:25.885927+0000","last_clean":"2023-07-06T01:20:25.885927+0000","last_became_active":"2023-07-06T01:20:23.853982+0000","last_became_peered":"2023-07-06T01:20:23.853982+0000","last_unstale":"2023-07-06T01:20:25.885927+0000","last_undegraded":"2023-07-06T01:20:25.885927+0000","last_fullsized":"2023-07-06T01:20:25.885927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:01.772125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836100+0000","last_change":"2023-07-06T01:19:19.586426+0000","last_active":"2023-07-06T01:20:25.836100+0000","last_peered":"2023-07-06T01:20:25.836100+0000","last_clean":"2023-07-06T01:20:25.836100+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:20:25.836100+0000","last_undegraded":"2023-07-06T01:20:25.836100+0000","last_fullsized":"2023-07-06T01:20:25.836100+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:48:48.093368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886966+0000","last_change":"2023-07-06T01:20:24.855685+0000","last_active":"2023-07-06T01:20:25.886966+0000","last_peered":"2023-07-06T01:20:25.886966+0000","last_clean":"2023-07-06T01:20:25.886966+0000","last_became_active":"2023-07-06T01:20:24.855347+0000","last_became_peered":"2023-07-06T01:20:24.855347+0000","last_unstale":"2023-07-06T01:20:25.886966+0000","last_undegraded":"2023-07-06T01:20:25.886966+0000","last_fullsized":"2023-07-06T01:20:25.886966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:39:36.610034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836192+0000","last_change":"2023-07-06T01:20:23.848203+0000","last_active":"2023-07-06T01:20:25.836192+0000","last_peered":"2023-07-06T01:20:25.836192+0000","last_clean":"2023-07-06T01:20:25.836192+0000","last_became_active":"2023-07-06T01:20:23.847910+0000","last_became_peered":"2023-07-06T01:20:23.847910+0000","last_unstale":"2023-07-06T01:20:25.836192+0000","last_undegraded":"2023-07-06T01:20:25.836192+0000","last_fullsized":"2023-07-06T01:20:25.836192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:59:43.797182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":39,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836647+0000","last_change":"2023-07-06T01:19:19.586626+0000","last_active":"2023-07-06T01:20:25.836647+0000","last_peered":"2023-07-06T01:20:25.836647+0000","last_clean":"2023-07-06T01:20:25.836647+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:20:25.836647+0000","last_undegraded":"2023-07-06T01:20:25.836647+0000","last_fullsized":"2023-07-06T01:20:25.836647+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:31:34.960495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.852493+0000","last_change":"2023-07-06T01:20:24.845956+0000","last_active":"2023-07-06T01:20:24.852493+0000","last_peered":"2023-07-06T01:20:24.852493+0000","last_clean":"2023-07-06T01:20:24.852493+0000","last_became_active":"2023-07-06T01:20:24.845698+0000","last_became_peered":"2023-07-06T01:20:24.845698+0000","last_unstale":"2023-07-06T01:20:24.852493+0000","last_undegraded":"2023-07-06T01:20:24.852493+0000","last_fullsized":"2023-07-06T01:20:24.852493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:41.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834898+0000","last_change":"2023-07-06T01:20:23.847902+0000","last_active":"2023-07-06T01:20:25.834898+0000","last_peered":"2023-07-06T01:20:25.834898+0000","last_clean":"2023-07-06T01:20:25.834898+0000","last_became_active":"2023-07-06T01:20:23.847543+0000","last_became_peered":"2023-07-06T01:20:23.847543+0000","last_unstale":"2023-07-06T01:20:25.834898+0000","last_undegraded":"2023-07-06T01:20:25.834898+0000","last_fullsized":"2023-07-06T01:20:25.834898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:51.462337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886549+0000","last_change":"2023-07-06T01:19:19.584086+0000","last_active":"2023-07-06T01:20:25.886549+0000","last_peered":"2023-07-06T01:20:25.886549+0000","last_clean":"2023-07-06T01:20:25.886549+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:20:25.886549+0000","last_undegraded":"2023-07-06T01:20:25.886549+0000","last_fullsized":"2023-07-06T01:20:25.886549+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:01:33.579869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855022+0000","last_change":"2023-07-06T01:20:24.851662+0000","last_active":"2023-07-06T01:20:24.855022+0000","last_peered":"2023-07-06T01:20:24.855022+0000","last_clean":"2023-07-06T01:20:24.855022+0000","last_became_active":"2023-07-06T01:20:24.851349+0000","last_became_peered":"2023-07-06T01:20:24.851349+0000","last_unstale":"2023-07-06T01:20:24.855022+0000","last_undegraded":"2023-07-06T01:20:24.855022+0000","last_fullsized":"2023-07-06T01:20:24.855022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:50.421447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836073+0000","last_change":"2023-07-06T01:20:23.845197+0000","last_active":"2023-07-06T01:20:25.836073+0000","last_peered":"2023-07-06T01:20:25.836073+0000","last_clean":"2023-07-06T01:20:25.836073+0000","last_became_active":"2023-07-06T01:20:23.844508+0000","last_became_peered":"2023-07-06T01:20:23.844508+0000","last_unstale":"2023-07-06T01:20:25.836073+0000","last_undegraded":"2023-07-06T01:20:25.836073+0000","last_fullsized":"2023-07-06T01:20:25.836073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:00.176807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836633+0000","last_change":"2023-07-06T01:19:19.586424+0000","last_active":"2023-07-06T01:20:25.836633+0000","last_peered":"2023-07-06T01:20:25.836633+0000","last_clean":"2023-07-06T01:20:25.836633+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:20:25.836633+0000","last_undegraded":"2023-07-06T01:20:25.836633+0000","last_fullsized":"2023-07-06T01:20:25.836633+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:04:57.410877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834442+0000","last_change":"2023-07-06T01:20:24.849984+0000","last_active":"2023-07-06T01:20:25.834442+0000","last_peered":"2023-07-06T01:20:25.834442+0000","last_clean":"2023-07-06T01:20:25.834442+0000","last_became_active":"2023-07-06T01:20:24.849603+0000","last_became_peered":"2023-07-06T01:20:24.849603+0000","last_unstale":"2023-07-06T01:20:25.834442+0000","last_undegraded":"2023-07-06T01:20:25.834442+0000","last_fullsized":"2023-07-06T01:20:25.834442+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:39:50.052979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834826+0000","last_change":"2023-07-06T01:20:23.842452+0000","last_active":"2023-07-06T01:20:25.834826+0000","last_peered":"2023-07-06T01:20:25.834826+0000","last_clean":"2023-07-06T01:20:25.834826+0000","last_became_active":"2023-07-06T01:20:23.842176+0000","last_became_peered":"2023-07-06T01:20:23.842176+0000","last_unstale":"2023-07-06T01:20:25.834826+0000","last_undegraded":"2023-07-06T01:20:25.834826+0000","last_fullsized":"2023-07-06T01:20:25.834826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:16.176519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836914+0000","last_change":"2023-07-06T01:19:19.583699+0000","last_active":"2023-07-06T01:20:25.836914+0000","last_peered":"2023-07-06T01:20:25.836914+0000","last_clean":"2023-07-06T01:20:25.836914+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:20:25.836914+0000","last_undegraded":"2023-07-06T01:20:25.836914+0000","last_fullsized":"2023-07-06T01:20:25.836914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:52:09.171519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":991232,"data_stored":749952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.504},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.63}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65800000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":158554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.71999999999999997}]},{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46600000000000003}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.74299999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.81200000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":966656,"data_stored":741755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":12.23,"5min":12.23,"15min":12.23},"last":0.311},{"interface":"front","average":{"1min":1.839,"5min":1.839,"15min":1.839},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":15.208,"5min":15.208,"15min":15.208},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":12.074,"5min":12.074,"15min":12.074},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":11.988,"5min":11.988,"15min":11.988},"last":0.73999999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.173,"5min":2.173,"15min":2.173},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"last":0.442},{"interface":"front","average":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":15.244999999999999,"5min":15.244999999999999,"15min":15.244999999999999},"last":0.379}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":11.923,"5min":11.923,"15min":11.923},"last":2.6080000000000001},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":12.019,"5min":12.019,"15min":12.019},"last":3.5950000000000002}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":11.999000000000001,"5min":11.999000000000001,"15min":11.999000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":11.951000000000001,"5min":11.951000000000001,"15min":11.951000000000001},"last":0.624}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":15.167999999999999,"5min":15.167999999999999,"15min":15.167999999999999},"last":2.7490000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":12.041,"5min":12.041,"15min":12.041},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":15.115,"5min":15.115,"15min":15.115},"last":1.625},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":11.965,"5min":11.965,"15min":11.965},"last":1.8200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":157984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":0.251}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.215},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.27200000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.503},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":6.3049999999999997,"5min":6.3049999999999997,"15min":6.3049999999999997},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.2430000000000003,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.63900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.436},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.625}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54900000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":152895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.307},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.375}]},{"osd":2,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.32700000000000001}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":3.8130000000000002,"5min":3.8130000000000002,"15min":3.8130000000000002},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.71899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":158062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.5},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.436}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":158532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.629},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.505},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.439}]},{"osd":5,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42399999999999999}]},{"osd":7,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:20:31.836 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:20:31.838 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:20:33.623 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:20:33.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.761+0000 7f2646e61700 1 Processor -- start 2023-07-06T01:20:33.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.761+0000 7f2646e61700 1 -- start start 2023-07-06T01:20:33.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f2646e61700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f26400cad00 0x7f26400cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:33.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f2646e61700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f264015cfd0 0x7f264015d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f2646e61700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264015d950 con 0x7f26401595d0 2023-07-06T01:20:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f2646e61700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26400cb6e0 con 0x7f26400cad00 2023-07-06T01:20:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f2646e61700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264006f3e0 con 0x7f264015cfd0 2023-07-06T01:20:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f263ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f26400cad00 0x7f26400cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f2644bfd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f26401595d0 0x7f26400cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38240/0 (socket says 172.21.15.22:38240) 2023-07-06T01:20:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f2644bfd700 1 -- 172.21.15.22:0/832395354 learned_addr learned my addr 172.21.15.22:0/832395354 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f26453fe700 1 --2- 172.21.15.22:0/832395354 >> v2:172.21.15.103:3300/0 conn(0x7f264015cfd0 0x7f264015d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f263ffff700 1 -- 172.21.15.22:0/832395354 >> v2:172.21.15.103:3300/0 conn(0x7f264015cfd0 msgr2=0x7f264015d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f263ffff700 1 --2- 172.21.15.22:0/832395354 >> v2:172.21.15.103:3300/0 conn(0x7f264015cfd0 0x7f264015d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f263ffff700 1 -- 172.21.15.22:0/832395354 >> v1:172.21.15.22:6789/0 conn(0x7f26401595d0 legacy=0x7f26400cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f263ffff700 1 -- 172.21.15.22:0/832395354 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264015dec0 con 0x7f26400cad00 2023-07-06T01:20:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f263f7fe700 1 -- 172.21.15.22:0/832395354 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272875932 0 0) 0x7f264015d950 con 0x7f26401595d0 2023-07-06T01:20:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.762+0000 7f26453fe700 1 --2- 172.21.15.22:0/832395354 >> v2:172.21.15.103:3300/0 conn(0x7f264015cfd0 0x7f264015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.763+0000 7f263ffff700 1 --2- 172.21.15.22:0/832395354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26400cad00 0x7f26400cb1a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f264006c410 tx=0x7f262c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc00e7c4eee02fb1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.763+0000 7f263f7fe700 1 -- 172.21.15.22:0/832395354 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c011830 con 0x7f26400cad00 2023-07-06T01:20:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.763+0000 7f263f7fe700 1 -- 172.21.15.22:0/832395354 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c011990 con 0x7f26400cad00 2023-07-06T01:20:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.763+0000 7f263f7fe700 1 -- 172.21.15.22:0/832395354 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c011d20 con 0x7f26400cad00 2023-07-06T01:20:33.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.763+0000 7f2646e61700 1 -- 172.21.15.22:0/832395354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26400cad00 msgr2=0x7f26400cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:33.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.763+0000 7f2646e61700 1 --2- 172.21.15.22:0/832395354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26400cad00 0x7f26400cb1a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f264006c410 tx=0x7f262c009b20 comp rx=0 tx=0).stop 2023-07-06T01:20:33.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.763+0000 7f2646e61700 1 -- 172.21.15.22:0/832395354 shutdown_connections 2023-07-06T01:20:33.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.763+0000 7f2646e61700 1 --2- 172.21.15.22:0/832395354 >> v2:172.21.15.103:3300/0 conn(0x7f264015cfd0 0x7f264015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.763+0000 7f2646e61700 1 --2- 172.21.15.22:0/832395354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26400cad00 0x7f26400cb1a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.763+0000 7f2646e61700 1 -- 172.21.15.22:0/832395354 >> 172.21.15.22:0/832395354 conn(0x7f26400b1890 msgr2=0x7f26400b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.763+0000 7f2646e61700 1 -- 172.21.15.22:0/832395354 shutdown_connections 2023-07-06T01:20:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.763+0000 7f2646e61700 1 -- 172.21.15.22:0/832395354 wait complete. 2023-07-06T01:20:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.765+0000 7f2646e61700 1 Processor -- start 2023-07-06T01:20:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.765+0000 7f2646e61700 1 -- start start 2023-07-06T01:20:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.765+0000 7f2646e61700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26400cad00 0x7f26400b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.765+0000 7f2646e61700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26401595d0 0x7f26400c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.765+0000 7f2646e61700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f264015cfd0 0x7f26400af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2646e61700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26400b0150 con 0x7f26400cad00 2023-07-06T01:20:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2646e61700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26400b0430 con 0x7f26401595d0 2023-07-06T01:20:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2646e61700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26400b0710 con 0x7f264015cfd0 2023-07-06T01:20:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2644bfd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26400cad00 0x7f26400b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2644bfd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26400cad00 0x7f26400b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47502/0 (socket says 172.21.15.22:47502) 2023-07-06T01:20:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2644bfd700 1 -- 172.21.15.22:0/1782303104 learned_addr learned my addr 172.21.15.22:0/1782303104 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f263ffff700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26401595d0 0x7f26400c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:33.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f26453fe700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f264015cfd0 0x7f26400af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:33.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2644bfd700 1 -- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f264015cfd0 msgr2=0x7f26400af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:33.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2644bfd700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f264015cfd0 0x7f26400af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:33.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2644bfd700 1 -- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26401595d0 msgr2=0x7f26400c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:33.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2644bfd700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26401595d0 0x7f26400c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:33.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2644bfd700 1 -- 172.21.15.22:0/1782303104 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c0094a0 con 0x7f26400cad00 2023-07-06T01:20:33.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f26453fe700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f264015cfd0 0x7f26400af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:20:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2644bfd700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26400cad00 0x7f26400b11f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f26340011e0 tx=0x7f263400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f263d7fa700 1 -- 172.21.15.22:0/1782303104 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634004c80 con 0x7f26400cad00 2023-07-06T01:20:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f263d7fa700 1 -- 172.21.15.22:0/1782303104 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2634004e00 con 0x7f26400cad00 2023-07-06T01:20:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2646e61700 1 -- 172.21.15.22:0/1782303104 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26400b09f0 con 0x7f26400cad00 2023-07-06T01:20:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f263d7fa700 1 -- 172.21.15.22:0/1782303104 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634007310 con 0x7f26400cad00 2023-07-06T01:20:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.766+0000 7f2646e61700 1 -- 172.21.15.22:0/1782303104 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26400c1310 con 0x7f26400cad00 2023-07-06T01:20:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.768+0000 7f2646e61700 1 -- 172.21.15.22:0/1782303104 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26400b0570 con 0x7f26400cad00 2023-07-06T01:20:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.768+0000 7f263d7fa700 1 -- 172.21.15.22:0/1782303104 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2634007470 con 0x7f26400cad00 2023-07-06T01:20:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.768+0000 7f263d7fa700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f262406ace0 0x7f262406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.768+0000 7f263d7fa700 1 -- 172.21.15.22:0/1782303104 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2634089190 con 0x7f26400cad00 2023-07-06T01:20:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.769+0000 7f263ffff700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f262406ace0 0x7f262406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.769+0000 7f263ffff700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f262406ace0 0x7f262406d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f262c0090b0 tx=0x7f262c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.773+0000 7f263d7fa700 1 -- 172.21.15.22:0/1782303104 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f26400b0570 con 0x7f26400cad00 2023-07-06T01:20:33.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.970+0000 7f2646e61700 1 -- 172.21.15.22:0/1782303104 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26400ce560 con 0x7f262406ace0 2023-07-06T01:20:33.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.979+0000 7f263d7fa700 1 -- 172.21.15.22:0/1782303104 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311210 (secure 0 0 0) 0x7f262c011cb0 con 0x7f262406ace0 2023-07-06T01:20:33.984 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:33.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.982+0000 7f2646e61700 1 -- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f262406ace0 msgr2=0x7f262406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:33.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.982+0000 7f2646e61700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f262406ace0 0x7f262406d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f262c0090b0 tx=0x7f262c008030 comp rx=0 tx=0).stop 2023-07-06T01:20:33.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.982+0000 7f2646e61700 1 -- 172.21.15.22:0/1782303104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26400cad00 msgr2=0x7f26400b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:33.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.982+0000 7f2646e61700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26400cad00 0x7f26400b11f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f26340011e0 tx=0x7f263400cae0 comp rx=0 tx=0).stop 2023-07-06T01:20:33.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.983+0000 7f2646e61700 1 -- 172.21.15.22:0/1782303104 shutdown_connections 2023-07-06T01:20:33.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.983+0000 7f2646e61700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f264015cfd0 0x7f26400af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:33.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.983+0000 7f2646e61700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f26400cad00 0x7f26400b11f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:33.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.983+0000 7f2646e61700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f262406ace0 0x7f262406d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:33.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.983+0000 7f2646e61700 1 --2- 172.21.15.22:0/1782303104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26401595d0 0x7f26400c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:33.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.983+0000 7f2646e61700 1 -- 172.21.15.22:0/1782303104 >> 172.21.15.22:0/1782303104 conn(0x7f26400b1890 msgr2=0x7f2640143360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:33.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.983+0000 7f2646e61700 1 -- 172.21.15.22:0/1782303104 shutdown_connections 2023-07-06T01:20:33.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:33.983+0000 7f2646e61700 1 -- 172.21.15.22:0/1782303104 wait complete. 2023-07-06T01:20:33.987 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:20:33.996 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-07-06T01:20:32.015493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219792,"kb_used_data":4176,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754928,"statfs":{"total":773094113280,"available":772869046272,"internally_reserved":0,"allocated":4276224,"data_stored":2435754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002033"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855626+0000","last_change":"2023-07-06T01:20:24.852545+0000","last_active":"2023-07-06T01:20:24.855626+0000","last_peered":"2023-07-06T01:20:24.855626+0000","last_clean":"2023-07-06T01:20:24.855626+0000","last_became_active":"2023-07-06T01:20:24.852249+0000","last_became_peered":"2023-07-06T01:20:24.852249+0000","last_unstale":"2023-07-06T01:20:24.855626+0000","last_undegraded":"2023-07-06T01:20:24.855626+0000","last_fullsized":"2023-07-06T01:20:24.855626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:17.181181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.854187+0000","last_change":"2023-07-06T01:20:24.850725+0000","last_active":"2023-07-06T01:20:24.854187+0000","last_peered":"2023-07-06T01:20:24.854187+0000","last_clean":"2023-07-06T01:20:24.854187+0000","last_became_active":"2023-07-06T01:20:24.849359+0000","last_became_peered":"2023-07-06T01:20:24.849359+0000","last_unstale":"2023-07-06T01:20:24.854187+0000","last_undegraded":"2023-07-06T01:20:24.854187+0000","last_fullsized":"2023-07-06T01:20:24.854187+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:20.409326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834789+0000","last_change":"2023-07-06T01:20:24.855847+0000","last_active":"2023-07-06T01:20:25.834789+0000","last_peered":"2023-07-06T01:20:25.834789+0000","last_clean":"2023-07-06T01:20:25.834789+0000","last_became_active":"2023-07-06T01:20:24.855571+0000","last_became_peered":"2023-07-06T01:20:24.855571+0000","last_unstale":"2023-07-06T01:20:25.834789+0000","last_undegraded":"2023-07-06T01:20:25.834789+0000","last_fullsized":"2023-07-06T01:20:25.834789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884281+0000","last_change":"2023-07-06T01:20:24.860708+0000","last_active":"2023-07-06T01:20:25.884281+0000","last_peered":"2023-07-06T01:20:25.884281+0000","last_clean":"2023-07-06T01:20:25.884281+0000","last_became_active":"2023-07-06T01:20:24.860220+0000","last_became_peered":"2023-07-06T01:20:24.860220+0000","last_unstale":"2023-07-06T01:20:25.884281+0000","last_undegraded":"2023-07-06T01:20:25.884281+0000","last_fullsized":"2023-07-06T01:20:25.884281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:42:33.239141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884127+0000","last_change":"2023-07-06T01:20:24.850342+0000","last_active":"2023-07-06T01:20:25.884127+0000","last_peered":"2023-07-06T01:20:25.884127+0000","last_clean":"2023-07-06T01:20:25.884127+0000","last_became_active":"2023-07-06T01:20:24.850083+0000","last_became_peered":"2023-07-06T01:20:24.850083+0000","last_unstale":"2023-07-06T01:20:25.884127+0000","last_undegraded":"2023-07-06T01:20:25.884127+0000","last_fullsized":"2023-07-06T01:20:25.884127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:46:35.434349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.850816+0000","last_change":"2023-07-06T01:20:24.848000+0000","last_active":"2023-07-06T01:20:24.850816+0000","last_peered":"2023-07-06T01:20:24.850816+0000","last_clean":"2023-07-06T01:20:24.850816+0000","last_became_active":"2023-07-06T01:20:24.847814+0000","last_became_peered":"2023-07-06T01:20:24.847814+0000","last_unstale":"2023-07-06T01:20:24.850816+0000","last_undegraded":"2023-07-06T01:20:24.850816+0000","last_fullsized":"2023-07-06T01:20:24.850816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:51.766923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886546+0000","last_change":"2023-07-06T01:20:24.856730+0000","last_active":"2023-07-06T01:20:25.886546+0000","last_peered":"2023-07-06T01:20:25.886546+0000","last_clean":"2023-07-06T01:20:25.886546+0000","last_became_active":"2023-07-06T01:20:24.856460+0000","last_became_peered":"2023-07-06T01:20:24.856460+0000","last_unstale":"2023-07-06T01:20:25.886546+0000","last_undegraded":"2023-07-06T01:20:25.886546+0000","last_fullsized":"2023-07-06T01:20:25.886546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:52.466998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859689+0000","last_change":"2023-07-06T01:20:24.858297+0000","last_active":"2023-07-06T01:20:24.859689+0000","last_peered":"2023-07-06T01:20:24.859689+0000","last_clean":"2023-07-06T01:20:24.859689+0000","last_became_active":"2023-07-06T01:20:24.858033+0000","last_became_peered":"2023-07-06T01:20:24.858033+0000","last_unstale":"2023-07-06T01:20:24.859689+0000","last_undegraded":"2023-07-06T01:20:24.859689+0000","last_fullsized":"2023-07-06T01:20:24.859689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:02:38.837442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834859+0000","last_change":"2023-07-06T01:20:24.859281+0000","last_active":"2023-07-06T01:20:25.834859+0000","last_peered":"2023-07-06T01:20:25.834859+0000","last_clean":"2023-07-06T01:20:25.834859+0000","last_became_active":"2023-07-06T01:20:24.858982+0000","last_became_peered":"2023-07-06T01:20:24.858982+0000","last_unstale":"2023-07-06T01:20:25.834859+0000","last_undegraded":"2023-07-06T01:20:25.834859+0000","last_fullsized":"2023-07-06T01:20:25.834859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:41.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886707+0000","last_change":"2023-07-06T01:20:24.854480+0000","last_active":"2023-07-06T01:20:25.886707+0000","last_peered":"2023-07-06T01:20:25.886707+0000","last_clean":"2023-07-06T01:20:25.886707+0000","last_became_active":"2023-07-06T01:20:24.854116+0000","last_became_peered":"2023-07-06T01:20:24.854116+0000","last_unstale":"2023-07-06T01:20:25.886707+0000","last_undegraded":"2023-07-06T01:20:25.886707+0000","last_fullsized":"2023-07-06T01:20:25.886707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:12.103457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886974+0000","last_change":"2023-07-06T01:20:24.854685+0000","last_active":"2023-07-06T01:20:25.886974+0000","last_peered":"2023-07-06T01:20:25.886974+0000","last_clean":"2023-07-06T01:20:25.886974+0000","last_became_active":"2023-07-06T01:20:24.853605+0000","last_became_peered":"2023-07-06T01:20:24.853605+0000","last_unstale":"2023-07-06T01:20:25.886974+0000","last_undegraded":"2023-07-06T01:20:25.886974+0000","last_fullsized":"2023-07-06T01:20:25.886974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:54:12.127760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834625+0000","last_change":"2023-07-06T01:20:24.853728+0000","last_active":"2023-07-06T01:20:25.834625+0000","last_peered":"2023-07-06T01:20:25.834625+0000","last_clean":"2023-07-06T01:20:25.834625+0000","last_became_active":"2023-07-06T01:20:24.853427+0000","last_became_peered":"2023-07-06T01:20:24.853427+0000","last_unstale":"2023-07-06T01:20:25.834625+0000","last_undegraded":"2023-07-06T01:20:25.834625+0000","last_fullsized":"2023-07-06T01:20:25.834625+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:31.557538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886831+0000","last_change":"2023-07-06T01:20:24.856012+0000","last_active":"2023-07-06T01:20:25.886831+0000","last_peered":"2023-07-06T01:20:25.886831+0000","last_clean":"2023-07-06T01:20:25.886831+0000","last_became_active":"2023-07-06T01:20:24.855762+0000","last_became_peered":"2023-07-06T01:20:24.855762+0000","last_unstale":"2023-07-06T01:20:25.886831+0000","last_undegraded":"2023-07-06T01:20:25.886831+0000","last_fullsized":"2023-07-06T01:20:25.886831+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:45.220808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.423022+0000","last_change":"2023-07-06T01:20:24.855945+0000","last_active":"2023-07-06T01:20:26.423022+0000","last_peered":"2023-07-06T01:20:26.423022+0000","last_clean":"2023-07-06T01:20:26.423022+0000","last_became_active":"2023-07-06T01:20:24.855312+0000","last_became_peered":"2023-07-06T01:20:24.855312+0000","last_unstale":"2023-07-06T01:20:26.423022+0000","last_undegraded":"2023-07-06T01:20:26.423022+0000","last_fullsized":"2023-07-06T01:20:26.423022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:08:48.865776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886397+0000","last_change":"2023-07-06T01:20:24.857423+0000","last_active":"2023-07-06T01:20:25.886397+0000","last_peered":"2023-07-06T01:20:25.886397+0000","last_clean":"2023-07-06T01:20:25.886397+0000","last_became_active":"2023-07-06T01:20:24.857164+0000","last_became_peered":"2023-07-06T01:20:24.857164+0000","last_unstale":"2023-07-06T01:20:25.886397+0000","last_undegraded":"2023-07-06T01:20:25.886397+0000","last_fullsized":"2023-07-06T01:20:25.886397+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836435+0000","last_change":"2023-07-06T01:20:24.858965+0000","last_active":"2023-07-06T01:20:25.836435+0000","last_peered":"2023-07-06T01:20:25.836435+0000","last_clean":"2023-07-06T01:20:25.836435+0000","last_became_active":"2023-07-06T01:20:24.858671+0000","last_became_peered":"2023-07-06T01:20:24.858671+0000","last_unstale":"2023-07-06T01:20:25.836435+0000","last_undegraded":"2023-07-06T01:20:25.836435+0000","last_fullsized":"2023-07-06T01:20:25.836435+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:02:07.880193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836250+0000","last_change":"2023-07-06T01:20:24.855887+0000","last_active":"2023-07-06T01:20:25.836250+0000","last_peered":"2023-07-06T01:20:25.836250+0000","last_clean":"2023-07-06T01:20:25.836250+0000","last_became_active":"2023-07-06T01:20:24.855621+0000","last_became_peered":"2023-07-06T01:20:24.855621+0000","last_unstale":"2023-07-06T01:20:25.836250+0000","last_undegraded":"2023-07-06T01:20:25.836250+0000","last_fullsized":"2023-07-06T01:20:25.836250+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:26:49.817004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836700+0000","last_change":"2023-07-06T01:20:24.860237+0000","last_active":"2023-07-06T01:20:25.836700+0000","last_peered":"2023-07-06T01:20:25.836700+0000","last_clean":"2023-07-06T01:20:25.836700+0000","last_became_active":"2023-07-06T01:20:24.859324+0000","last_became_peered":"2023-07-06T01:20:24.859324+0000","last_unstale":"2023-07-06T01:20:25.836700+0000","last_undegraded":"2023-07-06T01:20:25.836700+0000","last_fullsized":"2023-07-06T01:20:25.836700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:50.731526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884196+0000","last_change":"2023-07-06T01:20:24.852638+0000","last_active":"2023-07-06T01:20:25.884196+0000","last_peered":"2023-07-06T01:20:25.884196+0000","last_clean":"2023-07-06T01:20:25.884196+0000","last_became_active":"2023-07-06T01:20:24.852341+0000","last_became_peered":"2023-07-06T01:20:24.852341+0000","last_unstale":"2023-07-06T01:20:25.884196+0000","last_undegraded":"2023-07-06T01:20:25.884196+0000","last_fullsized":"2023-07-06T01:20:25.884196+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:17:34.632929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836872+0000","last_change":"2023-07-06T01:20:24.859626+0000","last_active":"2023-07-06T01:20:25.836872+0000","last_peered":"2023-07-06T01:20:25.836872+0000","last_clean":"2023-07-06T01:20:25.836872+0000","last_became_active":"2023-07-06T01:20:24.859274+0000","last_became_peered":"2023-07-06T01:20:24.859274+0000","last_unstale":"2023-07-06T01:20:25.836872+0000","last_undegraded":"2023-07-06T01:20:25.836872+0000","last_fullsized":"2023-07-06T01:20:25.836872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:03:24.140950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886696+0000","last_change":"2023-07-06T01:20:24.854443+0000","last_active":"2023-07-06T01:20:25.886696+0000","last_peered":"2023-07-06T01:20:25.886696+0000","last_clean":"2023-07-06T01:20:25.886696+0000","last_became_active":"2023-07-06T01:20:24.853256+0000","last_became_peered":"2023-07-06T01:20:24.853256+0000","last_unstale":"2023-07-06T01:20:25.886696+0000","last_undegraded":"2023-07-06T01:20:25.886696+0000","last_fullsized":"2023-07-06T01:20:25.886696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:20.409326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422484+0000","last_change":"2023-07-06T01:20:24.849709+0000","last_active":"2023-07-06T01:20:26.422484+0000","last_peered":"2023-07-06T01:20:26.422484+0000","last_clean":"2023-07-06T01:20:26.422484+0000","last_became_active":"2023-07-06T01:20:24.849426+0000","last_became_peered":"2023-07-06T01:20:24.849426+0000","last_unstale":"2023-07-06T01:20:26.422484+0000","last_undegraded":"2023-07-06T01:20:26.422484+0000","last_fullsized":"2023-07-06T01:20:26.422484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:15:43.509777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836223+0000","last_change":"2023-07-06T01:20:24.854999+0000","last_active":"2023-07-06T01:20:25.836223+0000","last_peered":"2023-07-06T01:20:25.836223+0000","last_clean":"2023-07-06T01:20:25.836223+0000","last_became_active":"2023-07-06T01:20:24.854710+0000","last_became_peered":"2023-07-06T01:20:24.854710+0000","last_unstale":"2023-07-06T01:20:25.836223+0000","last_undegraded":"2023-07-06T01:20:25.836223+0000","last_fullsized":"2023-07-06T01:20:25.836223+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:35:50.376971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884398+0000","last_change":"2023-07-06T01:20:24.852693+0000","last_active":"2023-07-06T01:20:25.884398+0000","last_peered":"2023-07-06T01:20:25.884398+0000","last_clean":"2023-07-06T01:20:25.884398+0000","last_became_active":"2023-07-06T01:20:24.852395+0000","last_became_peered":"2023-07-06T01:20:24.852395+0000","last_unstale":"2023-07-06T01:20:25.884398+0000","last_undegraded":"2023-07-06T01:20:25.884398+0000","last_fullsized":"2023-07-06T01:20:25.884398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:35.585237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836707+0000","last_change":"2023-07-06T01:20:24.856089+0000","last_active":"2023-07-06T01:20:25.836707+0000","last_peered":"2023-07-06T01:20:25.836707+0000","last_clean":"2023-07-06T01:20:25.836707+0000","last_became_active":"2023-07-06T01:20:24.855834+0000","last_became_peered":"2023-07-06T01:20:24.855834+0000","last_unstale":"2023-07-06T01:20:25.836707+0000","last_undegraded":"2023-07-06T01:20:25.836707+0000","last_fullsized":"2023-07-06T01:20:25.836707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:03.093141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859016+0000","last_change":"2023-07-06T01:20:24.856754+0000","last_active":"2023-07-06T01:20:24.859016+0000","last_peered":"2023-07-06T01:20:24.859016+0000","last_clean":"2023-07-06T01:20:24.859016+0000","last_became_active":"2023-07-06T01:20:24.856396+0000","last_became_peered":"2023-07-06T01:20:24.856396+0000","last_unstale":"2023-07-06T01:20:24.859016+0000","last_undegraded":"2023-07-06T01:20:24.859016+0000","last_fullsized":"2023-07-06T01:20:24.859016+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:06:02.668449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835952+0000","last_change":"2023-07-06T01:20:24.854996+0000","last_active":"2023-07-06T01:20:25.835952+0000","last_peered":"2023-07-06T01:20:25.835952+0000","last_clean":"2023-07-06T01:20:25.835952+0000","last_became_active":"2023-07-06T01:20:24.854737+0000","last_became_peered":"2023-07-06T01:20:24.854737+0000","last_unstale":"2023-07-06T01:20:25.835952+0000","last_undegraded":"2023-07-06T01:20:25.835952+0000","last_fullsized":"2023-07-06T01:20:25.835952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:30:51.766923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886641+0000","last_change":"2023-07-06T01:20:24.856804+0000","last_active":"2023-07-06T01:20:25.886641+0000","last_peered":"2023-07-06T01:20:25.886641+0000","last_clean":"2023-07-06T01:20:25.886641+0000","last_became_active":"2023-07-06T01:20:24.856490+0000","last_became_peered":"2023-07-06T01:20:24.856490+0000","last_unstale":"2023-07-06T01:20:25.886641+0000","last_undegraded":"2023-07-06T01:20:25.886641+0000","last_fullsized":"2023-07-06T01:20:25.886641+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:23:07.815642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835630+0000","last_change":"2023-07-06T01:20:24.850799+0000","last_active":"2023-07-06T01:20:25.835630+0000","last_peered":"2023-07-06T01:20:25.835630+0000","last_clean":"2023-07-06T01:20:25.835630+0000","last_became_active":"2023-07-06T01:20:24.850463+0000","last_became_peered":"2023-07-06T01:20:24.850463+0000","last_unstale":"2023-07-06T01:20:25.835630+0000","last_undegraded":"2023-07-06T01:20:25.835630+0000","last_fullsized":"2023-07-06T01:20:25.835630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:50.421447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835944+0000","last_change":"2023-07-06T01:20:24.857969+0000","last_active":"2023-07-06T01:20:25.835944+0000","last_peered":"2023-07-06T01:20:25.835944+0000","last_clean":"2023-07-06T01:20:25.835944+0000","last_became_active":"2023-07-06T01:20:24.857674+0000","last_became_peered":"2023-07-06T01:20:24.857674+0000","last_unstale":"2023-07-06T01:20:25.835944+0000","last_undegraded":"2023-07-06T01:20:25.835944+0000","last_fullsized":"2023-07-06T01:20:25.835944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:57.090630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835732+0000","last_change":"2023-07-06T01:20:24.850856+0000","last_active":"2023-07-06T01:20:25.835732+0000","last_peered":"2023-07-06T01:20:25.835732+0000","last_clean":"2023-07-06T01:20:25.835732+0000","last_became_active":"2023-07-06T01:20:24.850484+0000","last_became_peered":"2023-07-06T01:20:24.850484+0000","last_unstale":"2023-07-06T01:20:25.835732+0000","last_undegraded":"2023-07-06T01:20:25.835732+0000","last_fullsized":"2023-07-06T01:20:25.835732+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:38:28.819319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.856018+0000","last_change":"2023-07-06T01:20:24.854034+0000","last_active":"2023-07-06T01:20:24.856018+0000","last_peered":"2023-07-06T01:20:24.856018+0000","last_clean":"2023-07-06T01:20:24.856018+0000","last_became_active":"2023-07-06T01:20:24.853820+0000","last_became_peered":"2023-07-06T01:20:24.853820+0000","last_unstale":"2023-07-06T01:20:24.856018+0000","last_undegraded":"2023-07-06T01:20:24.856018+0000","last_fullsized":"2023-07-06T01:20:24.856018+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:38:57.090630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836081+0000","last_change":"2023-07-06T01:20:24.857755+0000","last_active":"2023-07-06T01:20:25.836081+0000","last_peered":"2023-07-06T01:20:25.836081+0000","last_clean":"2023-07-06T01:20:25.836081+0000","last_became_active":"2023-07-06T01:20:24.855354+0000","last_became_peered":"2023-07-06T01:20:24.855354+0000","last_unstale":"2023-07-06T01:20:25.836081+0000","last_undegraded":"2023-07-06T01:20:25.836081+0000","last_fullsized":"2023-07-06T01:20:25.836081+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:55:10.623339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836884+0000","last_change":"2023-07-06T01:20:23.849203+0000","last_active":"2023-07-06T01:20:25.836884+0000","last_peered":"2023-07-06T01:20:25.836884+0000","last_clean":"2023-07-06T01:20:25.836884+0000","last_became_active":"2023-07-06T01:20:23.848883+0000","last_became_peered":"2023-07-06T01:20:23.848883+0000","last_unstale":"2023-07-06T01:20:25.836884+0000","last_undegraded":"2023-07-06T01:20:25.836884+0000","last_fullsized":"2023-07-06T01:20:25.836884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:06.770007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.860470+0000","last_change":"2023-07-06T01:20:24.859359+0000","last_active":"2023-07-06T01:20:24.860470+0000","last_peered":"2023-07-06T01:20:24.860470+0000","last_clean":"2023-07-06T01:20:24.860470+0000","last_became_active":"2023-07-06T01:20:24.859152+0000","last_became_peered":"2023-07-06T01:20:24.859152+0000","last_unstale":"2023-07-06T01:20:24.860470+0000","last_undegraded":"2023-07-06T01:20:24.860470+0000","last_fullsized":"2023-07-06T01:20:24.860470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:01.181469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834689+0000","last_change":"2023-07-06T01:20:23.845980+0000","last_active":"2023-07-06T01:20:25.834689+0000","last_peered":"2023-07-06T01:20:25.834689+0000","last_clean":"2023-07-06T01:20:25.834689+0000","last_became_active":"2023-07-06T01:20:23.845696+0000","last_became_peered":"2023-07-06T01:20:23.845696+0000","last_unstale":"2023-07-06T01:20:25.834689+0000","last_undegraded":"2023-07-06T01:20:25.834689+0000","last_fullsized":"2023-07-06T01:20:25.834689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:58.579906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422869+0000","last_change":"2023-07-06T01:20:24.851207+0000","last_active":"2023-07-06T01:20:26.422869+0000","last_peered":"2023-07-06T01:20:26.422869+0000","last_clean":"2023-07-06T01:20:26.422869+0000","last_became_active":"2023-07-06T01:20:24.850929+0000","last_became_peered":"2023-07-06T01:20:24.850929+0000","last_unstale":"2023-07-06T01:20:26.422869+0000","last_undegraded":"2023-07-06T01:20:26.422869+0000","last_fullsized":"2023-07-06T01:20:26.422869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:38:16.612537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.858947+0000","last_change":"2023-07-06T01:20:23.846742+0000","last_active":"2023-07-06T01:20:24.858947+0000","last_peered":"2023-07-06T01:20:24.858947+0000","last_clean":"2023-07-06T01:20:24.858947+0000","last_became_active":"2023-07-06T01:20:23.846440+0000","last_became_peered":"2023-07-06T01:20:23.846440+0000","last_unstale":"2023-07-06T01:20:24.858947+0000","last_undegraded":"2023-07-06T01:20:24.858947+0000","last_fullsized":"2023-07-06T01:20:24.858947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:17:06.770007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862194+0000","last_change":"2023-07-06T01:20:24.861558+0000","last_active":"2023-07-06T01:20:24.862194+0000","last_peered":"2023-07-06T01:20:24.862194+0000","last_clean":"2023-07-06T01:20:24.862194+0000","last_became_active":"2023-07-06T01:20:24.860857+0000","last_became_peered":"2023-07-06T01:20:24.860857+0000","last_unstale":"2023-07-06T01:20:24.862194+0000","last_undegraded":"2023-07-06T01:20:24.862194+0000","last_fullsized":"2023-07-06T01:20:24.862194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:52.466998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422998+0000","last_change":"2023-07-06T01:20:23.848100+0000","last_active":"2023-07-06T01:20:26.422998+0000","last_peered":"2023-07-06T01:20:26.422998+0000","last_clean":"2023-07-06T01:20:26.422998+0000","last_became_active":"2023-07-06T01:20:23.847541+0000","last_became_peered":"2023-07-06T01:20:23.847541+0000","last_unstale":"2023-07-06T01:20:26.422998+0000","last_undegraded":"2023-07-06T01:20:26.422998+0000","last_fullsized":"2023-07-06T01:20:26.422998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:35:05.552930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834373+0000","last_change":"2023-07-06T01:20:24.839527+0000","last_active":"2023-07-06T01:20:25.834373+0000","last_peered":"2023-07-06T01:20:25.834373+0000","last_clean":"2023-07-06T01:20:25.834373+0000","last_became_active":"2023-07-06T01:20:24.839201+0000","last_became_peered":"2023-07-06T01:20:24.839201+0000","last_unstale":"2023-07-06T01:20:25.834373+0000","last_undegraded":"2023-07-06T01:20:25.834373+0000","last_fullsized":"2023-07-06T01:20:25.834373+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:57:20.290658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886526+0000","last_change":"2023-07-06T01:20:23.853509+0000","last_active":"2023-07-06T01:20:25.886526+0000","last_peered":"2023-07-06T01:20:25.886526+0000","last_clean":"2023-07-06T01:20:25.886526+0000","last_became_active":"2023-07-06T01:20:23.853060+0000","last_became_peered":"2023-07-06T01:20:23.853060+0000","last_unstale":"2023-07-06T01:20:25.886526+0000","last_undegraded":"2023-07-06T01:20:25.886526+0000","last_fullsized":"2023-07-06T01:20:25.886526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:40.214108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886380+0000","last_change":"2023-07-06T01:20:24.856240+0000","last_active":"2023-07-06T01:20:25.886380+0000","last_peered":"2023-07-06T01:20:25.886380+0000","last_clean":"2023-07-06T01:20:25.886380+0000","last_became_active":"2023-07-06T01:20:24.853426+0000","last_became_peered":"2023-07-06T01:20:24.853426+0000","last_unstale":"2023-07-06T01:20:25.886380+0000","last_undegraded":"2023-07-06T01:20:25.886380+0000","last_fullsized":"2023-07-06T01:20:25.886380+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:59.584568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836264+0000","last_change":"2023-07-06T01:20:23.853484+0000","last_active":"2023-07-06T01:20:25.836264+0000","last_peered":"2023-07-06T01:20:25.836264+0000","last_clean":"2023-07-06T01:20:25.836264+0000","last_became_active":"2023-07-06T01:20:23.853173+0000","last_became_peered":"2023-07-06T01:20:23.853173+0000","last_unstale":"2023-07-06T01:20:25.836264+0000","last_undegraded":"2023-07-06T01:20:25.836264+0000","last_fullsized":"2023-07-06T01:20:25.836264+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:55:15.600714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422525+0000","last_change":"2023-07-06T01:20:24.856110+0000","last_active":"2023-07-06T01:20:26.422525+0000","last_peered":"2023-07-06T01:20:26.422525+0000","last_clean":"2023-07-06T01:20:26.422525+0000","last_became_active":"2023-07-06T01:20:24.855686+0000","last_became_peered":"2023-07-06T01:20:24.855686+0000","last_unstale":"2023-07-06T01:20:26.422525+0000","last_undegraded":"2023-07-06T01:20:26.422525+0000","last_fullsized":"2023-07-06T01:20:26.422525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:04:28.309392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836572+0000","last_change":"2023-07-06T01:20:23.849040+0000","last_active":"2023-07-06T01:20:24.836572+0000","last_peered":"2023-07-06T01:20:24.836572+0000","last_clean":"2023-07-06T01:20:24.836572+0000","last_became_active":"2023-07-06T01:20:23.848756+0000","last_became_peered":"2023-07-06T01:20:23.848756+0000","last_unstale":"2023-07-06T01:20:24.836572+0000","last_undegraded":"2023-07-06T01:20:24.836572+0000","last_fullsized":"2023-07-06T01:20:24.836572+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:05:00.225120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855846+0000","last_change":"2023-07-06T01:20:24.849751+0000","last_active":"2023-07-06T01:20:24.855846+0000","last_peered":"2023-07-06T01:20:24.855846+0000","last_clean":"2023-07-06T01:20:24.855846+0000","last_became_active":"2023-07-06T01:20:24.849475+0000","last_became_peered":"2023-07-06T01:20:24.849475+0000","last_unstale":"2023-07-06T01:20:24.855846+0000","last_undegraded":"2023-07-06T01:20:24.855846+0000","last_fullsized":"2023-07-06T01:20:24.855846+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:45:53.117974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422290+0000","last_change":"2023-07-06T01:20:23.842790+0000","last_active":"2023-07-06T01:20:26.422290+0000","last_peered":"2023-07-06T01:20:26.422290+0000","last_clean":"2023-07-06T01:20:26.422290+0000","last_became_active":"2023-07-06T01:20:23.842488+0000","last_became_peered":"2023-07-06T01:20:23.842488+0000","last_unstale":"2023-07-06T01:20:26.422290+0000","last_undegraded":"2023-07-06T01:20:26.422290+0000","last_fullsized":"2023-07-06T01:20:26.422290+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:17:05.908517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884424+0000","last_change":"2023-07-06T01:20:24.860540+0000","last_active":"2023-07-06T01:20:25.884424+0000","last_peered":"2023-07-06T01:20:25.884424+0000","last_clean":"2023-07-06T01:20:25.884424+0000","last_became_active":"2023-07-06T01:20:24.859869+0000","last_became_peered":"2023-07-06T01:20:24.859869+0000","last_unstale":"2023-07-06T01:20:25.884424+0000","last_undegraded":"2023-07-06T01:20:25.884424+0000","last_fullsized":"2023-07-06T01:20:25.884424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:53:23.281099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.845141+0000","last_change":"2023-07-06T01:20:23.848957+0000","last_active":"2023-07-06T01:20:24.845141+0000","last_peered":"2023-07-06T01:20:24.845141+0000","last_clean":"2023-07-06T01:20:24.845141+0000","last_became_active":"2023-07-06T01:20:23.848646+0000","last_became_peered":"2023-07-06T01:20:23.848646+0000","last_unstale":"2023-07-06T01:20:24.845141+0000","last_undegraded":"2023-07-06T01:20:24.845141+0000","last_fullsized":"2023-07-06T01:20:24.845141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:35:50.683696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836461+0000","last_change":"2023-07-06T01:20:24.853193+0000","last_active":"2023-07-06T01:20:25.836461+0000","last_peered":"2023-07-06T01:20:25.836461+0000","last_clean":"2023-07-06T01:20:25.836461+0000","last_became_active":"2023-07-06T01:20:24.852778+0000","last_became_peered":"2023-07-06T01:20:24.852778+0000","last_unstale":"2023-07-06T01:20:25.836461+0000","last_undegraded":"2023-07-06T01:20:25.836461+0000","last_fullsized":"2023-07-06T01:20:25.836461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:30:29.768004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886834+0000","last_change":"2023-07-06T01:20:23.847858+0000","last_active":"2023-07-06T01:20:25.886834+0000","last_peered":"2023-07-06T01:20:25.886834+0000","last_clean":"2023-07-06T01:20:25.886834+0000","last_became_active":"2023-07-06T01:20:23.847549+0000","last_became_peered":"2023-07-06T01:20:23.847549+0000","last_unstale":"2023-07-06T01:20:25.886834+0000","last_undegraded":"2023-07-06T01:20:25.886834+0000","last_fullsized":"2023-07-06T01:20:25.886834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:53:56.761538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834565+0000","last_change":"2023-07-06T01:20:24.848275+0000","last_active":"2023-07-06T01:20:25.834565+0000","last_peered":"2023-07-06T01:20:25.834565+0000","last_clean":"2023-07-06T01:20:25.834565+0000","last_became_active":"2023-07-06T01:20:24.847949+0000","last_became_peered":"2023-07-06T01:20:24.847949+0000","last_unstale":"2023-07-06T01:20:25.834565+0000","last_undegraded":"2023-07-06T01:20:25.834565+0000","last_fullsized":"2023-07-06T01:20:25.834565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:24:03.775615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887068+0000","last_change":"2023-07-06T01:20:23.849878+0000","last_active":"2023-07-06T01:20:25.887068+0000","last_peered":"2023-07-06T01:20:25.887068+0000","last_clean":"2023-07-06T01:20:25.887068+0000","last_became_active":"2023-07-06T01:20:23.849573+0000","last_became_peered":"2023-07-06T01:20:23.849573+0000","last_unstale":"2023-07-06T01:20:25.887068+0000","last_undegraded":"2023-07-06T01:20:25.887068+0000","last_fullsized":"2023-07-06T01:20:25.887068+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:12:26.209870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422768+0000","last_change":"2023-07-06T01:20:24.854277+0000","last_active":"2023-07-06T01:20:26.422768+0000","last_peered":"2023-07-06T01:20:26.422768+0000","last_clean":"2023-07-06T01:20:26.422768+0000","last_became_active":"2023-07-06T01:20:24.853981+0000","last_became_peered":"2023-07-06T01:20:24.853981+0000","last_unstale":"2023-07-06T01:20:26.422768+0000","last_undegraded":"2023-07-06T01:20:26.422768+0000","last_fullsized":"2023-07-06T01:20:26.422768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:59:32.105014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.868009+0000","last_change":"2023-07-06T01:20:23.845648+0000","last_active":"2023-07-06T01:20:24.868009+0000","last_peered":"2023-07-06T01:20:24.868009+0000","last_clean":"2023-07-06T01:20:24.868009+0000","last_became_active":"2023-07-06T01:20:23.845340+0000","last_became_peered":"2023-07-06T01:20:23.845340+0000","last_unstale":"2023-07-06T01:20:24.868009+0000","last_undegraded":"2023-07-06T01:20:24.868009+0000","last_fullsized":"2023-07-06T01:20:24.868009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:39:35.605373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834521+0000","last_change":"2023-07-06T01:20:24.858282+0000","last_active":"2023-07-06T01:20:25.834521+0000","last_peered":"2023-07-06T01:20:25.834521+0000","last_clean":"2023-07-06T01:20:25.834521+0000","last_became_active":"2023-07-06T01:20:24.857879+0000","last_became_peered":"2023-07-06T01:20:24.857879+0000","last_unstale":"2023-07-06T01:20:25.834521+0000","last_undegraded":"2023-07-06T01:20:25.834521+0000","last_fullsized":"2023-07-06T01:20:25.834521+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:22:46.902965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855788+0000","last_change":"2023-07-06T01:20:23.841745+0000","last_active":"2023-07-06T01:20:24.855788+0000","last_peered":"2023-07-06T01:20:24.855788+0000","last_clean":"2023-07-06T01:20:24.855788+0000","last_became_active":"2023-07-06T01:20:23.841438+0000","last_became_peered":"2023-07-06T01:20:23.841438+0000","last_unstale":"2023-07-06T01:20:24.855788+0000","last_undegraded":"2023-07-06T01:20:24.855788+0000","last_fullsized":"2023-07-06T01:20:24.855788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:54:11.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855590+0000","last_change":"2023-07-06T01:20:24.854272+0000","last_active":"2023-07-06T01:20:24.855590+0000","last_peered":"2023-07-06T01:20:24.855590+0000","last_clean":"2023-07-06T01:20:24.855590+0000","last_became_active":"2023-07-06T01:20:24.854002+0000","last_became_peered":"2023-07-06T01:20:24.854002+0000","last_unstale":"2023-07-06T01:20:24.855590+0000","last_undegraded":"2023-07-06T01:20:24.855590+0000","last_fullsized":"2023-07-06T01:20:24.855590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:23:07.815642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836795+0000","last_change":"2023-07-06T01:20:23.854212+0000","last_active":"2023-07-06T01:20:24.836795+0000","last_peered":"2023-07-06T01:20:24.836795+0000","last_clean":"2023-07-06T01:20:24.836795+0000","last_became_active":"2023-07-06T01:20:23.853928+0000","last_became_peered":"2023-07-06T01:20:23.853928+0000","last_unstale":"2023-07-06T01:20:24.836795+0000","last_undegraded":"2023-07-06T01:20:24.836795+0000","last_fullsized":"2023-07-06T01:20:24.836795+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:06:49.726864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835989+0000","last_change":"2023-07-06T01:20:24.855007+0000","last_active":"2023-07-06T01:20:25.835989+0000","last_peered":"2023-07-06T01:20:25.835989+0000","last_clean":"2023-07-06T01:20:25.835989+0000","last_became_active":"2023-07-06T01:20:24.854610+0000","last_became_peered":"2023-07-06T01:20:24.854610+0000","last_unstale":"2023-07-06T01:20:25.835989+0000","last_undegraded":"2023-07-06T01:20:25.835989+0000","last_fullsized":"2023-07-06T01:20:25.835989+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:27:38.804964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835969+0000","last_change":"2023-07-06T01:20:23.847446+0000","last_active":"2023-07-06T01:20:25.835969+0000","last_peered":"2023-07-06T01:20:25.835969+0000","last_clean":"2023-07-06T01:20:25.835969+0000","last_became_active":"2023-07-06T01:20:23.846929+0000","last_became_peered":"2023-07-06T01:20:23.846929+0000","last_unstale":"2023-07-06T01:20:25.835969+0000","last_undegraded":"2023-07-06T01:20:25.835969+0000","last_fullsized":"2023-07-06T01:20:25.835969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:05:00.225120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855866+0000","last_change":"2023-07-06T01:20:24.855191+0000","last_active":"2023-07-06T01:20:24.855866+0000","last_peered":"2023-07-06T01:20:24.855866+0000","last_clean":"2023-07-06T01:20:24.855866+0000","last_became_active":"2023-07-06T01:20:24.854898+0000","last_became_peered":"2023-07-06T01:20:24.854898+0000","last_unstale":"2023-07-06T01:20:24.855866+0000","last_undegraded":"2023-07-06T01:20:24.855866+0000","last_fullsized":"2023-07-06T01:20:24.855866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.419832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'8","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836552+0000","last_change":"2023-07-06T01:20:23.853463+0000","last_active":"2023-07-06T01:20:25.836552+0000","last_peered":"2023-07-06T01:20:25.836552+0000","last_clean":"2023-07-06T01:20:25.836552+0000","last_became_active":"2023-07-06T01:20:23.853104+0000","last_became_peered":"2023-07-06T01:20:23.853104+0000","last_unstale":"2023-07-06T01:20:25.836552+0000","last_undegraded":"2023-07-06T01:20:25.836552+0000","last_fullsized":"2023-07-06T01:20:25.836552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:59:04.523315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862166+0000","last_change":"2023-07-06T01:20:24.861072+0000","last_active":"2023-07-06T01:20:24.862166+0000","last_peered":"2023-07-06T01:20:24.862166+0000","last_clean":"2023-07-06T01:20:24.862166+0000","last_became_active":"2023-07-06T01:20:24.860745+0000","last_became_peered":"2023-07-06T01:20:24.860745+0000","last_unstale":"2023-07-06T01:20:24.862166+0000","last_undegraded":"2023-07-06T01:20:24.862166+0000","last_fullsized":"2023-07-06T01:20:24.862166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:07:45.220808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887003+0000","last_change":"2023-07-06T01:20:23.848801+0000","last_active":"2023-07-06T01:20:25.887003+0000","last_peered":"2023-07-06T01:20:25.887003+0000","last_clean":"2023-07-06T01:20:25.887003+0000","last_became_active":"2023-07-06T01:20:23.848412+0000","last_became_peered":"2023-07-06T01:20:23.848412+0000","last_unstale":"2023-07-06T01:20:25.887003+0000","last_undegraded":"2023-07-06T01:20:25.887003+0000","last_fullsized":"2023-07-06T01:20:25.887003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:35:50.683696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886952+0000","last_change":"2023-07-06T01:20:24.854853+0000","last_active":"2023-07-06T01:20:25.886952+0000","last_peered":"2023-07-06T01:20:25.886952+0000","last_clean":"2023-07-06T01:20:25.886952+0000","last_became_active":"2023-07-06T01:20:24.854159+0000","last_became_peered":"2023-07-06T01:20:24.854159+0000","last_unstale":"2023-07-06T01:20:25.886952+0000","last_undegraded":"2023-07-06T01:20:25.886952+0000","last_fullsized":"2023-07-06T01:20:25.886952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:04:12.419832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422967+0000","last_change":"2023-07-06T01:20:23.850450+0000","last_active":"2023-07-06T01:20:26.422967+0000","last_peered":"2023-07-06T01:20:26.422967+0000","last_clean":"2023-07-06T01:20:26.422967+0000","last_became_active":"2023-07-06T01:20:23.850157+0000","last_became_peered":"2023-07-06T01:20:23.850157+0000","last_unstale":"2023-07-06T01:20:26.422967+0000","last_undegraded":"2023-07-06T01:20:26.422967+0000","last_fullsized":"2023-07-06T01:20:26.422967+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:03:07.971132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422880+0000","last_change":"2023-07-06T01:20:24.843242+0000","last_active":"2023-07-06T01:20:26.422880+0000","last_peered":"2023-07-06T01:20:26.422880+0000","last_clean":"2023-07-06T01:20:26.422880+0000","last_became_active":"2023-07-06T01:20:24.842910+0000","last_became_peered":"2023-07-06T01:20:24.842910+0000","last_unstale":"2023-07-06T01:20:26.422880+0000","last_undegraded":"2023-07-06T01:20:26.422880+0000","last_fullsized":"2023-07-06T01:20:26.422880+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:11:05.585857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884278+0000","last_change":"2023-07-06T01:20:23.846883+0000","last_active":"2023-07-06T01:20:25.884278+0000","last_peered":"2023-07-06T01:20:25.884278+0000","last_clean":"2023-07-06T01:20:25.884278+0000","last_became_active":"2023-07-06T01:20:23.846566+0000","last_became_peered":"2023-07-06T01:20:23.846566+0000","last_unstale":"2023-07-06T01:20:25.884278+0000","last_undegraded":"2023-07-06T01:20:25.884278+0000","last_fullsized":"2023-07-06T01:20:25.884278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:44:49.870273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884313+0000","last_change":"2023-07-06T01:20:24.853783+0000","last_active":"2023-07-06T01:20:25.884313+0000","last_peered":"2023-07-06T01:20:25.884313+0000","last_clean":"2023-07-06T01:20:25.884313+0000","last_became_active":"2023-07-06T01:20:24.853427+0000","last_became_peered":"2023-07-06T01:20:24.853427+0000","last_unstale":"2023-07-06T01:20:25.884313+0000","last_undegraded":"2023-07-06T01:20:25.884313+0000","last_fullsized":"2023-07-06T01:20:25.884313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:33:25.311596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.837321+0000","last_change":"2023-07-06T01:20:23.847016+0000","last_active":"2023-07-06T01:20:24.837321+0000","last_peered":"2023-07-06T01:20:24.837321+0000","last_clean":"2023-07-06T01:20:24.837321+0000","last_became_active":"2023-07-06T01:20:23.846748+0000","last_became_peered":"2023-07-06T01:20:23.846748+0000","last_unstale":"2023-07-06T01:20:24.837321+0000","last_undegraded":"2023-07-06T01:20:24.837321+0000","last_fullsized":"2023-07-06T01:20:24.837321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:53:13.424430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836548+0000","last_change":"2023-07-06T01:20:24.855376+0000","last_active":"2023-07-06T01:20:25.836548+0000","last_peered":"2023-07-06T01:20:25.836548+0000","last_clean":"2023-07-06T01:20:25.836548+0000","last_became_active":"2023-07-06T01:20:24.855107+0000","last_became_peered":"2023-07-06T01:20:24.855107+0000","last_unstale":"2023-07-06T01:20:25.836548+0000","last_undegraded":"2023-07-06T01:20:25.836548+0000","last_fullsized":"2023-07-06T01:20:25.836548+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:42:46.861502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884268+0000","last_change":"2023-07-06T01:20:23.847882+0000","last_active":"2023-07-06T01:20:25.884268+0000","last_peered":"2023-07-06T01:20:25.884268+0000","last_clean":"2023-07-06T01:20:25.884268+0000","last_became_active":"2023-07-06T01:20:23.847534+0000","last_became_peered":"2023-07-06T01:20:23.847534+0000","last_unstale":"2023-07-06T01:20:25.884268+0000","last_undegraded":"2023-07-06T01:20:25.884268+0000","last_fullsized":"2023-07-06T01:20:25.884268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:22:12.321678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834687+0000","last_change":"2023-07-06T01:20:24.848033+0000","last_active":"2023-07-06T01:20:25.834687+0000","last_peered":"2023-07-06T01:20:25.834687+0000","last_clean":"2023-07-06T01:20:25.834687+0000","last_became_active":"2023-07-06T01:20:24.847514+0000","last_became_peered":"2023-07-06T01:20:24.847514+0000","last_unstale":"2023-07-06T01:20:25.834687+0000","last_undegraded":"2023-07-06T01:20:25.834687+0000","last_fullsized":"2023-07-06T01:20:25.834687+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:01:02.805903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884363+0000","last_change":"2023-07-06T01:20:23.850960+0000","last_active":"2023-07-06T01:20:25.884363+0000","last_peered":"2023-07-06T01:20:25.884363+0000","last_clean":"2023-07-06T01:20:25.884363+0000","last_became_active":"2023-07-06T01:20:23.850699+0000","last_became_peered":"2023-07-06T01:20:23.850699+0000","last_unstale":"2023-07-06T01:20:25.884363+0000","last_undegraded":"2023-07-06T01:20:25.884363+0000","last_fullsized":"2023-07-06T01:20:25.884363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:30:39.863076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886519+0000","last_change":"2023-07-06T01:20:24.861071+0000","last_active":"2023-07-06T01:20:25.886519+0000","last_peered":"2023-07-06T01:20:25.886519+0000","last_clean":"2023-07-06T01:20:25.886519+0000","last_became_active":"2023-07-06T01:20:24.860803+0000","last_became_peered":"2023-07-06T01:20:24.860803+0000","last_unstale":"2023-07-06T01:20:25.886519+0000","last_undegraded":"2023-07-06T01:20:25.886519+0000","last_fullsized":"2023-07-06T01:20:25.886519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:17.181181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836165+0000","last_change":"2023-07-06T01:20:23.851273+0000","last_active":"2023-07-06T01:20:25.836165+0000","last_peered":"2023-07-06T01:20:25.836165+0000","last_clean":"2023-07-06T01:20:25.836165+0000","last_became_active":"2023-07-06T01:20:23.850942+0000","last_became_peered":"2023-07-06T01:20:23.850942+0000","last_unstale":"2023-07-06T01:20:25.836165+0000","last_undegraded":"2023-07-06T01:20:25.836165+0000","last_fullsized":"2023-07-06T01:20:25.836165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:01:23.333657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.862183+0000","last_change":"2023-07-06T01:20:24.861226+0000","last_active":"2023-07-06T01:20:24.862183+0000","last_peered":"2023-07-06T01:20:24.862183+0000","last_clean":"2023-07-06T01:20:24.862183+0000","last_became_active":"2023-07-06T01:20:24.860944+0000","last_became_peered":"2023-07-06T01:20:24.860944+0000","last_unstale":"2023-07-06T01:20:24.862183+0000","last_undegraded":"2023-07-06T01:20:24.862183+0000","last_fullsized":"2023-07-06T01:20:24.862183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:21:12.103457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.884125+0000","last_change":"2023-07-06T01:20:23.849860+0000","last_active":"2023-07-06T01:20:25.884125+0000","last_peered":"2023-07-06T01:20:25.884125+0000","last_clean":"2023-07-06T01:20:25.884125+0000","last_became_active":"2023-07-06T01:20:23.849507+0000","last_became_peered":"2023-07-06T01:20:23.849507+0000","last_unstale":"2023-07-06T01:20:25.884125+0000","last_undegraded":"2023-07-06T01:20:25.884125+0000","last_fullsized":"2023-07-06T01:20:25.884125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:05:51.413216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.423002+0000","last_change":"2023-07-06T01:20:24.853501+0000","last_active":"2023-07-06T01:20:26.423002+0000","last_peered":"2023-07-06T01:20:26.423002+0000","last_clean":"2023-07-06T01:20:26.423002+0000","last_became_active":"2023-07-06T01:20:24.853232+0000","last_became_peered":"2023-07-06T01:20:24.853232+0000","last_unstale":"2023-07-06T01:20:26.423002+0000","last_undegraded":"2023-07-06T01:20:26.423002+0000","last_fullsized":"2023-07-06T01:20:26.423002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T13:14:59.968797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422341+0000","last_change":"2023-07-06T01:20:23.844519+0000","last_active":"2023-07-06T01:20:26.422341+0000","last_peered":"2023-07-06T01:20:26.422341+0000","last_clean":"2023-07-06T01:20:26.422341+0000","last_became_active":"2023-07-06T01:20:23.844260+0000","last_became_peered":"2023-07-06T01:20:23.844260+0000","last_unstale":"2023-07-06T01:20:26.422341+0000","last_undegraded":"2023-07-06T01:20:26.422341+0000","last_fullsized":"2023-07-06T01:20:26.422341+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:18:17.854150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:26.422622+0000","last_change":"2023-07-06T01:19:20.074195+0000","last_active":"2023-07-06T01:20:26.422622+0000","last_peered":"2023-07-06T01:20:26.422622+0000","last_clean":"2023-07-06T01:20:26.422622+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:20:26.422622+0000","last_undegraded":"2023-07-06T01:20:26.422622+0000","last_fullsized":"2023-07-06T01:20:26.422622+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:50:47.542825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.835909+0000","last_change":"2023-07-06T01:20:24.856069+0000","last_active":"2023-07-06T01:20:25.835909+0000","last_peered":"2023-07-06T01:20:25.835909+0000","last_clean":"2023-07-06T01:20:25.835909+0000","last_became_active":"2023-07-06T01:20:24.855766+0000","last_became_peered":"2023-07-06T01:20:24.855766+0000","last_unstale":"2023-07-06T01:20:25.835909+0000","last_undegraded":"2023-07-06T01:20:25.835909+0000","last_fullsized":"2023-07-06T01:20:25.835909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:48:55.581029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.842093+0000","last_change":"2023-07-06T01:20:23.849570+0000","last_active":"2023-07-06T01:20:24.842093+0000","last_peered":"2023-07-06T01:20:24.842093+0000","last_clean":"2023-07-06T01:20:24.842093+0000","last_became_active":"2023-07-06T01:20:23.849255+0000","last_became_peered":"2023-07-06T01:20:23.849255+0000","last_unstale":"2023-07-06T01:20:24.842093+0000","last_undegraded":"2023-07-06T01:20:24.842093+0000","last_fullsized":"2023-07-06T01:20:24.842093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:53:56.761538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.846124+0000","last_change":"2023-07-06T01:19:19.587110+0000","last_active":"2023-07-06T01:20:24.846124+0000","last_peered":"2023-07-06T01:20:24.846124+0000","last_clean":"2023-07-06T01:20:24.846124+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:20:24.846124+0000","last_undegraded":"2023-07-06T01:20:24.846124+0000","last_fullsized":"2023-07-06T01:20:24.846124+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:49:50.820241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.859472+0000","last_change":"2023-07-06T01:20:24.858252+0000","last_active":"2023-07-06T01:20:24.859472+0000","last_peered":"2023-07-06T01:20:24.859472+0000","last_clean":"2023-07-06T01:20:24.859472+0000","last_became_active":"2023-07-06T01:20:24.856691+0000","last_became_peered":"2023-07-06T01:20:24.856691+0000","last_unstale":"2023-07-06T01:20:24.859472+0000","last_undegraded":"2023-07-06T01:20:24.859472+0000","last_fullsized":"2023-07-06T01:20:24.859472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:52:03.093141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.887051+0000","last_change":"2023-07-06T01:20:23.847756+0000","last_active":"2023-07-06T01:20:25.887051+0000","last_peered":"2023-07-06T01:20:25.887051+0000","last_clean":"2023-07-06T01:20:25.887051+0000","last_became_active":"2023-07-06T01:20:23.847392+0000","last_became_peered":"2023-07-06T01:20:23.847392+0000","last_unstale":"2023-07-06T01:20:25.887051+0000","last_undegraded":"2023-07-06T01:20:25.887051+0000","last_fullsized":"2023-07-06T01:20:25.887051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:50:55.073152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":35,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.836206+0000","last_change":"2023-07-06T01:19:19.584242+0000","last_active":"2023-07-06T01:20:24.836206+0000","last_peered":"2023-07-06T01:20:24.836206+0000","last_clean":"2023-07-06T01:20:24.836206+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:20:24.836206+0000","last_undegraded":"2023-07-06T01:20:24.836206+0000","last_fullsized":"2023-07-06T01:20:24.836206+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:02:18.883377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":189,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836706+0000","last_change":"2023-07-06T01:19:19.586495+0000","last_active":"2023-07-06T01:20:25.836706+0000","last_peered":"2023-07-06T01:20:25.836706+0000","last_clean":"2023-07-06T01:20:25.836706+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:20:25.836706+0000","last_undegraded":"2023-07-06T01:20:25.836706+0000","last_fullsized":"2023-07-06T01:20:25.836706+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:13:17.664579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836620+0000","last_change":"2023-07-06T01:20:24.847523+0000","last_active":"2023-07-06T01:20:25.836620+0000","last_peered":"2023-07-06T01:20:25.836620+0000","last_clean":"2023-07-06T01:20:25.836620+0000","last_became_active":"2023-07-06T01:20:24.847256+0000","last_became_peered":"2023-07-06T01:20:24.847256+0000","last_unstale":"2023-07-06T01:20:25.836620+0000","last_undegraded":"2023-07-06T01:20:25.836620+0000","last_fullsized":"2023-07-06T01:20:25.836620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:26:53.631677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"34'1","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.885927+0000","last_change":"2023-07-06T01:20:23.854385+0000","last_active":"2023-07-06T01:20:25.885927+0000","last_peered":"2023-07-06T01:20:25.885927+0000","last_clean":"2023-07-06T01:20:25.885927+0000","last_became_active":"2023-07-06T01:20:23.853982+0000","last_became_peered":"2023-07-06T01:20:23.853982+0000","last_unstale":"2023-07-06T01:20:25.885927+0000","last_undegraded":"2023-07-06T01:20:25.885927+0000","last_fullsized":"2023-07-06T01:20:25.885927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:09:01.772125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836100+0000","last_change":"2023-07-06T01:19:19.586426+0000","last_active":"2023-07-06T01:20:25.836100+0000","last_peered":"2023-07-06T01:20:25.836100+0000","last_clean":"2023-07-06T01:20:25.836100+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:20:25.836100+0000","last_undegraded":"2023-07-06T01:20:25.836100+0000","last_fullsized":"2023-07-06T01:20:25.836100+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:48:48.093368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886966+0000","last_change":"2023-07-06T01:20:24.855685+0000","last_active":"2023-07-06T01:20:25.886966+0000","last_peered":"2023-07-06T01:20:25.886966+0000","last_clean":"2023-07-06T01:20:25.886966+0000","last_became_active":"2023-07-06T01:20:24.855347+0000","last_became_peered":"2023-07-06T01:20:24.855347+0000","last_unstale":"2023-07-06T01:20:25.886966+0000","last_undegraded":"2023-07-06T01:20:25.886966+0000","last_fullsized":"2023-07-06T01:20:25.886966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:39:36.610034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836192+0000","last_change":"2023-07-06T01:20:23.848203+0000","last_active":"2023-07-06T01:20:25.836192+0000","last_peered":"2023-07-06T01:20:25.836192+0000","last_clean":"2023-07-06T01:20:25.836192+0000","last_became_active":"2023-07-06T01:20:23.847910+0000","last_became_peered":"2023-07-06T01:20:23.847910+0000","last_unstale":"2023-07-06T01:20:25.836192+0000","last_undegraded":"2023-07-06T01:20:25.836192+0000","last_fullsized":"2023-07-06T01:20:25.836192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T12:59:43.797182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":39,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836647+0000","last_change":"2023-07-06T01:19:19.586626+0000","last_active":"2023-07-06T01:20:25.836647+0000","last_peered":"2023-07-06T01:20:25.836647+0000","last_clean":"2023-07-06T01:20:25.836647+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:20:25.836647+0000","last_undegraded":"2023-07-06T01:20:25.836647+0000","last_fullsized":"2023-07-06T01:20:25.836647+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:31:34.960495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.852493+0000","last_change":"2023-07-06T01:20:24.845956+0000","last_active":"2023-07-06T01:20:24.852493+0000","last_peered":"2023-07-06T01:20:24.852493+0000","last_clean":"2023-07-06T01:20:24.852493+0000","last_became_active":"2023-07-06T01:20:24.845698+0000","last_became_peered":"2023-07-06T01:20:24.845698+0000","last_unstale":"2023-07-06T01:20:24.852493+0000","last_undegraded":"2023-07-06T01:20:24.852493+0000","last_fullsized":"2023-07-06T01:20:24.852493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:35:41.218769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834898+0000","last_change":"2023-07-06T01:20:23.847902+0000","last_active":"2023-07-06T01:20:25.834898+0000","last_peered":"2023-07-06T01:20:25.834898+0000","last_clean":"2023-07-06T01:20:25.834898+0000","last_became_active":"2023-07-06T01:20:23.847543+0000","last_became_peered":"2023-07-06T01:20:23.847543+0000","last_unstale":"2023-07-06T01:20:25.834898+0000","last_undegraded":"2023-07-06T01:20:25.834898+0000","last_fullsized":"2023-07-06T01:20:25.834898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:10:51.462337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":39,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.886549+0000","last_change":"2023-07-06T01:19:19.584086+0000","last_active":"2023-07-06T01:20:25.886549+0000","last_peered":"2023-07-06T01:20:25.886549+0000","last_clean":"2023-07-06T01:20:25.886549+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:20:25.886549+0000","last_undegraded":"2023-07-06T01:20:25.886549+0000","last_fullsized":"2023-07-06T01:20:25.886549+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:01:33.579869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-07-06T01:20:24.855022+0000","last_change":"2023-07-06T01:20:24.851662+0000","last_active":"2023-07-06T01:20:24.855022+0000","last_peered":"2023-07-06T01:20:24.855022+0000","last_clean":"2023-07-06T01:20:24.855022+0000","last_became_active":"2023-07-06T01:20:24.851349+0000","last_became_peered":"2023-07-06T01:20:24.851349+0000","last_unstale":"2023-07-06T01:20:24.855022+0000","last_undegraded":"2023-07-06T01:20:24.855022+0000","last_fullsized":"2023-07-06T01:20:24.855022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:07:50.421447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836073+0000","last_change":"2023-07-06T01:20:23.845197+0000","last_active":"2023-07-06T01:20:25.836073+0000","last_peered":"2023-07-06T01:20:25.836073+0000","last_clean":"2023-07-06T01:20:25.836073+0000","last_became_active":"2023-07-06T01:20:23.844508+0000","last_became_peered":"2023-07-06T01:20:23.844508+0000","last_unstale":"2023-07-06T01:20:25.836073+0000","last_undegraded":"2023-07-06T01:20:25.836073+0000","last_fullsized":"2023-07-06T01:20:25.836073+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T08:53:00.176807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836633+0000","last_change":"2023-07-06T01:19:19.586424+0000","last_active":"2023-07-06T01:20:25.836633+0000","last_peered":"2023-07-06T01:20:25.836633+0000","last_clean":"2023-07-06T01:20:25.836633+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:20:25.836633+0000","last_undegraded":"2023-07-06T01:20:25.836633+0000","last_fullsized":"2023-07-06T01:20:25.836633+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:04:57.410877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834442+0000","last_change":"2023-07-06T01:20:24.849984+0000","last_active":"2023-07-06T01:20:25.834442+0000","last_peered":"2023-07-06T01:20:25.834442+0000","last_clean":"2023-07-06T01:20:25.834442+0000","last_became_active":"2023-07-06T01:20:24.849603+0000","last_became_peered":"2023-07-06T01:20:24.849603+0000","last_unstale":"2023-07-06T01:20:25.834442+0000","last_undegraded":"2023-07-06T01:20:25.834442+0000","last_fullsized":"2023-07-06T01:20:25.834442+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:23.824595+0000","last_clean_scrub_stamp":"2023-07-06T01:20:23.824595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T02:39:50.052979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.834826+0000","last_change":"2023-07-06T01:20:23.842452+0000","last_active":"2023-07-06T01:20:25.834826+0000","last_peered":"2023-07-06T01:20:25.834826+0000","last_clean":"2023-07-06T01:20:25.834826+0000","last_became_active":"2023-07-06T01:20:23.842176+0000","last_became_peered":"2023-07-06T01:20:23.842176+0000","last_unstale":"2023-07-06T01:20:25.834826+0000","last_undegraded":"2023-07-06T01:20:25.834826+0000","last_fullsized":"2023-07-06T01:20:25.834826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:20:22.819933+0000","last_clean_scrub_stamp":"2023-07-06T01:20:22.819933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:10:16.176519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-07-06T01:20:25.836914+0000","last_change":"2023-07-06T01:19:19.583699+0000","last_active":"2023-07-06T01:20:25.836914+0000","last_peered":"2023-07-06T01:20:25.836914+0000","last_clean":"2023-07-06T01:20:25.836914+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:20:25.836914+0000","last_undegraded":"2023-07-06T01:20:25.836914+0000","last_fullsized":"2023-07-06T01:20:25.836914+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:52:09.171519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":991232,"data_stored":749952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.504},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.63}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65800000000000003}]}]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":158554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.71999999999999997}]},{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46600000000000003}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.74299999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.81200000000000006}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":966656,"data_stored":741755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.492,"15min":1.492},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":12.23,"5min":12.23,"15min":12.23},"last":0.311},{"interface":"front","average":{"1min":1.839,"5min":1.839,"15min":1.839},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":15.208,"5min":15.208,"15min":15.208},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":12.074,"5min":12.074,"15min":12.074},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":11.988,"5min":11.988,"15min":11.988},"last":0.73999999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.173,"5min":2.173,"15min":2.173},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":18.963000000000001,"5min":18.963000000000001,"15min":18.963000000000001},"last":0.442},{"interface":"front","average":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":15.244999999999999,"5min":15.244999999999999,"15min":15.244999999999999},"last":0.379}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":11.923,"5min":11.923,"15min":11.923},"last":2.6080000000000001},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":12.019,"5min":12.019,"15min":12.019},"last":3.5950000000000002}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":11.999000000000001,"5min":11.999000000000001,"15min":11.999000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":11.951000000000001,"5min":11.951000000000001,"15min":11.951000000000001},"last":0.624}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":15.167999999999999,"5min":15.167999999999999,"15min":15.167999999999999},"last":2.7490000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":12.041,"5min":12.041,"15min":12.041},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":15.115,"5min":15.115,"15min":15.115},"last":1.625},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":11.965,"5min":11.965,"15min":11.965},"last":1.8200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":157984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":0.251}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.215},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.27200000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.097,"5min":2.097,"15min":2.097},"last":0.503},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":6.3049999999999997,"5min":6.3049999999999997,"15min":6.3049999999999997},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.2430000000000003,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.63900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":158020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.436,"5min":0.436,"15min":0.436},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.36299999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.625}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54900000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":152895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.307},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.375}]},{"osd":2,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.32700000000000001}]},{"osd":4,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":3.8130000000000002,"5min":3.8130000000000002,"15min":3.8130000000000002},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jul 6 01:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.71899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":158062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.5},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.436}]},{"osd":7,"last update":"Thu Jul 6 01:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":385024,"data_stored":158532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.629},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.505},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.439}]},{"osd":5,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42399999999999999}]},{"osd":7,"last update":"Thu Jul 6 01:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:20:34.003 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T01:20:34.003 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T01:20:34.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.166+0000 7f7b0a7e2700 1 Processor -- start 2023-07-06T01:20:34.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.166+0000 7f7b0a7e2700 1 -- start start 2023-07-06T01:20:34.169 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.167+0000 7f7b0a7e2700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7b04165700 0x7f7b04167ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.167+0000 7f7b0a7e2700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b040715f0 con 0x7f7b0406b2b0 2023-07-06T01:20:34.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.167+0000 7f7b0a7e2700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0406d4e0 con 0x7f7b04070a90 2023-07-06T01:20:34.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.167+0000 7f7b0a7e2700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b041610a0 con 0x7f7b04165700 2023-07-06T01:20:34.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.167+0000 7f7b03fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f7b0406b2b0 0x7f7b0406b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38254/0 (socket says 172.21.15.22:38254) 2023-07-06T01:20:34.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.167+0000 7f7b03fff700 1 -- 172.21.15.22:0/339999481 learned_addr learned my addr 172.21.15.22:0/339999481 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:34.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.167+0000 7f7b08d7f700 1 --2- 172.21.15.22:0/339999481 >> v2:172.21.15.103:3300/0 conn(0x7f7b04165700 0x7f7b04167ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.167+0000 7f7b02ffd700 1 -- 172.21.15.22:0/339999481 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287184167 0 0) 0x7f7b040715f0 con 0x7f7b0406b2b0 2023-07-06T01:20:34.170 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.167+0000 7f7b02ffd700 1 -- 172.21.15.22:0/339999481 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7af00035b0 con 0x7f7b0406b2b0 2023-07-06T01:20:34.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b08d7f700 1 -- 172.21.15.22:0/339999481 >> v1:172.21.15.103:6789/0 conn(0x7f7b04070a90 legacy=0x7f7b04070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:34.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b08d7f700 1 -- 172.21.15.22:0/339999481 >> v1:172.21.15.22:6789/0 conn(0x7f7b0406b2b0 legacy=0x7f7b0406b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:34.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b08d7f700 1 -- 172.21.15.22:0/339999481 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b04160420 con 0x7f7b04165700 2023-07-06T01:20:34.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b08d7f700 1 --2- 172.21.15.22:0/339999481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b04165700 0x7f7b04167ae0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009d90 tx=0x7f7afc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34b6062ce1c78960 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:34.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b02ffd700 1 -- 172.21.15.22:0/339999481 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00e070 con 0x7f7b04165700 2023-07-06T01:20:34.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b02ffd700 1 -- 172.21.15.22:0/339999481 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc00bb20 con 0x7f7b04165700 2023-07-06T01:20:34.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b02ffd700 1 -- 172.21.15.22:0/339999481 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00be90 con 0x7f7b04165700 2023-07-06T01:20:34.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/339999481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b04165700 msgr2=0x7f7b04167ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:34.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b0a7e2700 1 --2- 172.21.15.22:0/339999481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b04165700 0x7f7b04167ae0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009d90 tx=0x7f7afc00b330 comp rx=0 tx=0).stop 2023-07-06T01:20:34.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/339999481 shutdown_connections 2023-07-06T01:20:34.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b0a7e2700 1 --2- 172.21.15.22:0/339999481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b04165700 0x7f7b04167ae0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/339999481 >> 172.21.15.22:0/339999481 conn(0x7f7b040cc710 msgr2=0x7f7b040ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:34.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/339999481 shutdown_connections 2023-07-06T01:20:34.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.168+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/339999481 wait complete. 2023-07-06T01:20:34.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.170+0000 7f7b0a7e2700 1 Processor -- start 2023-07-06T01:20:34.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.170+0000 7f7b0a7e2700 1 -- start start 2023-07-06T01:20:34.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b0a7e2700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b0406b2b0 0x7f7b041501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b0a7e2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b04070a90 0x7f7b040ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b0a7e2700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b04165700 0x7f7b040ad730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b0a7e2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040b2ac0 con 0x7f7b04070a90 2023-07-06T01:20:34.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b0a7e2700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0415e620 con 0x7f7b0406b2b0 2023-07-06T01:20:34.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b0a7e2700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04156c10 con 0x7f7b04165700 2023-07-06T01:20:34.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b04070a90 0x7f7b040ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b04070a90 0x7f7b040ad1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47516/0 (socket says 172.21.15.22:47516) 2023-07-06T01:20:34.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b03fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b0406b2b0 0x7f7b041501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b037fe700 1 -- 172.21.15.22:0/899873352 learned_addr learned my addr 172.21.15.22:0/899873352 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:34.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b08d7f700 1 --2- 172.21.15.22:0/899873352 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b04165700 0x7f7b040ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b037fe700 1 -- 172.21.15.22:0/899873352 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b04165700 msgr2=0x7f7b040ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:34.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b037fe700 1 --2- 172.21.15.22:0/899873352 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b04165700 0x7f7b040ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b037fe700 1 -- 172.21.15.22:0/899873352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b0406b2b0 msgr2=0x7f7b041501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:34.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b037fe700 1 --2- 172.21.15.22:0/899873352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b0406b2b0 0x7f7b041501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b037fe700 1 -- 172.21.15.22:0/899873352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc009a70 con 0x7f7b04070a90 2023-07-06T01:20:34.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b037fe700 1 --2- 172.21.15.22:0/899873352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b04070a90 0x7f7b040ad1f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c4c0 tx=0x7f7aec00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:34.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b00ff9700 1 -- 172.21.15.22:0/899873352 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0052d0 con 0x7f7b04070a90 2023-07-06T01:20:34.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b00ff9700 1 -- 172.21.15.22:0/899873352 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec00ce90 con 0x7f7b04070a90 2023-07-06T01:20:34.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b00ff9700 1 -- 172.21.15.22:0/899873352 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec006860 con 0x7f7b04070a90 2023-07-06T01:20:34.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.171+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/899873352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b041506e0 con 0x7f7b04070a90 2023-07-06T01:20:34.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.172+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/899873352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0415dfd0 con 0x7f7b04070a90 2023-07-06T01:20:34.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.172+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/899873352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b040744b0 con 0x7f7b04070a90 2023-07-06T01:20:34.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.173+0000 7f7b00ff9700 1 -- 172.21.15.22:0/899873352 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7aec003680 con 0x7f7b04070a90 2023-07-06T01:20:34.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.174+0000 7f7b00ff9700 1 --2- 172.21.15.22:0/899873352 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7ae406ace0 0x7f7ae406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.174+0000 7f7b00ff9700 1 -- 172.21.15.22:0/899873352 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7aec0886d0 con 0x7f7b04070a90 2023-07-06T01:20:34.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.174+0000 7f7b03fff700 1 --2- 172.21.15.22:0/899873352 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7ae406ace0 0x7f7ae406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.174+0000 7f7b03fff700 1 --2- 172.21.15.22:0/899873352 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7ae406ace0 0x7f7ae406d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7af4007fd0 tx=0x7f7af4007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:34.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.179+0000 7f7b00ff9700 1 -- 172.21.15.22:0/899873352 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7aec0536f0 con 0x7f7b04070a90 2023-07-06T01:20:34.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.457+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/899873352 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7b0415e760 con 0x7f7b04070a90 2023-07-06T01:20:34.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.461+0000 7f7b00ff9700 1 -- 172.21.15.22:0/899873352 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7aec056d10 con 0x7f7b04070a90 2023-07-06T01:20:34.463 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2023-07-06T01:20:34.463 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBqFqZkwHh1IRAAx8BVLni20wF8yL0mJjooBQ== 2023-07-06T01:20:34.463 INFO:teuthology.orchestra.run.smithi022.stdout: caps mds = "allow" 2023-07-06T01:20:34.463 INFO:teuthology.orchestra.run.smithi022.stdout: caps mon = "allow r" 2023-07-06T01:20:34.463 INFO:teuthology.orchestra.run.smithi022.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-07-06T01:20:34.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.463+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/899873352 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7ae406ace0 msgr2=0x7f7ae406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:34.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.463+0000 7f7b0a7e2700 1 --2- 172.21.15.22:0/899873352 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7ae406ace0 0x7f7ae406d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7af4007fd0 tx=0x7f7af4007dc0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.463+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/899873352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b04070a90 msgr2=0x7f7b040ad1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:34.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.463+0000 7f7b0a7e2700 1 --2- 172.21.15.22:0/899873352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b04070a90 0x7f7b040ad1f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c4c0 tx=0x7f7aec00c7d0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.465 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.463+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/899873352 shutdown_connections 2023-07-06T01:20:34.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.463+0000 7f7b0a7e2700 1 --2- 172.21.15.22:0/899873352 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b04165700 0x7f7b040ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.463+0000 7f7b0a7e2700 1 --2- 172.21.15.22:0/899873352 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7ae406ace0 0x7f7ae406d190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.463+0000 7f7b0a7e2700 1 --2- 172.21.15.22:0/899873352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b0406b2b0 0x7f7b041501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.463+0000 7f7b0a7e2700 1 --2- 172.21.15.22:0/899873352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b04070a90 0x7f7b040ad1f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.463+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/899873352 >> 172.21.15.22:0/899873352 conn(0x7f7b040cc710 msgr2=0x7f7b04151520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:34.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.463+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/899873352 shutdown_connections 2023-07-06T01:20:34.466 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.463+0000 7f7b0a7e2700 1 -- 172.21.15.22:0/899873352 wait complete. 2023-07-06T01:20:34.467 INFO:teuthology.orchestra.run.smithi022.stderr:updated caps for client.0 2023-07-06T01:20:34.477 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:20:34.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.599+0000 7f80b73dc700 1 Processor -- start 2023-07-06T01:20:34.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.600+0000 7f80b73dc700 1 -- start start 2023-07-06T01:20:34.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.600+0000 7f80b73dc700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f80b015a710 0x7f80b00ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.600+0000 7f80b73dc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80b00cef70 0x7f80b00cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.600+0000 7f80b73dc700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b00af270 con 0x7f80b00aecb0 2023-07-06T01:20:34.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80b73dc700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b00af550 con 0x7f80b015a710 2023-07-06T01:20:34.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80b73dc700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00af830 con 0x7f80b00cef70 2023-07-06T01:20:34.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80b4977700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f80b00aecb0 0x7f80b00ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38266/0 (socket says 172.21.15.22:38266) 2023-07-06T01:20:34.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80b4977700 1 -- 172.21.15.22:0/3134653546 learned_addr learned my addr 172.21.15.22:0/3134653546 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:34.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80b5178700 1 --2- 172.21.15.22:0/3134653546 >> v2:172.21.15.103:3300/0 conn(0x7f80b015a710 0x7f80b00ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80b5979700 1 --2- 172.21.15.22:0/3134653546 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80b00cef70 0x7f80b00cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1277901539 0 0) 0x7f80b00af270 con 0x7f80b00aecb0 2023-07-06T01:20:34.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f809c0035b0 con 0x7f80b00aecb0 2023-07-06T01:20:34.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 688981585 0 0) 0x7f809c0035b0 con 0x7f80b00aecb0 2023-07-06T01:20:34.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80b00cef70 msgr2=0x7f80b00cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:34.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 --2- 172.21.15.22:0/3134653546 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80b00cef70 0x7f80b00cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 >> v2:172.21.15.103:3300/0 conn(0x7f80b015a710 msgr2=0x7f80b00ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:34.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 --2- 172.21.15.22:0/3134653546 >> v2:172.21.15.103:3300/0 conn(0x7f80b015a710 0x7f80b00ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b00cb2e0 con 0x7f80b00aecb0 2023-07-06T01:20:34.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f80a4001670 con 0x7f80b00aecb0 2023-07-06T01:20:34.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 >> v1:172.21.15.22:6789/0 conn(0x7f80b00aecb0 legacy=0x7f80b00ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:34.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 --2- 172.21.15.22:0/3134653546 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80b00cef70 0x7f809c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 --2- 172.21.15.22:0/3134653546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f809c003f80 0x7f809c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 --2- 172.21.15.22:0/3134653546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f809c004d90 0x7f809c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00af550 con 0x7f809c004d90 2023-07-06T01:20:34.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00af830 con 0x7f809c003f80 2023-07-06T01:20:34.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c00ba10 con 0x7f80b00cef70 2023-07-06T01:20:34.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80b5979700 1 --2- 172.21.15.22:0/3134653546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f809c004d90 0x7f809c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.601+0000 7f80b5178700 1 --2- 172.21.15.22:0/3134653546 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80b00cef70 0x7f809c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.602+0000 7f80b4977700 1 --2- 172.21.15.22:0/3134653546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f809c003f80 0x7f809c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.602+0000 7f80b5979700 1 -- 172.21.15.22:0/3134653546 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80b00cef70 msgr2=0x7f809c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:34.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.602+0000 7f80b5979700 1 --2- 172.21.15.22:0/3134653546 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80b00cef70 0x7f809c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.602+0000 7f80b5979700 1 -- 172.21.15.22:0/3134653546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f809c003f80 msgr2=0x7f809c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:34.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.602+0000 7f80b5979700 1 --2- 172.21.15.22:0/3134653546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f809c003f80 0x7f809c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.602+0000 7f80b5979700 1 -- 172.21.15.22:0/3134653546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c0035b0 con 0x7f809c004d90 2023-07-06T01:20:34.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.602+0000 7f80b5979700 1 --2- 172.21.15.22:0/3134653546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f809c004d90 0x7f809c005200 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009fc0 tx=0x7f809c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840028b0bce06776 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:34.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.602+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a800ec30 con 0x7f809c004d90 2023-07-06T01:20:34.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.602+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a800f040 con 0x7f809c004d90 2023-07-06T01:20:34.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.602+0000 7f80a3fff700 1 -- 172.21.15.22:0/3134653546 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8014480 con 0x7f809c004d90 2023-07-06T01:20:34.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.602+0000 7f80b73dc700 1 -- 172.21.15.22:0/3134653546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f809c004d90 msgr2=0x7f809c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:34.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.602+0000 7f80b73dc700 1 --2- 172.21.15.22:0/3134653546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f809c004d90 0x7f809c005200 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009fc0 tx=0x7f809c005880 comp rx=0 tx=0).stop 2023-07-06T01:20:34.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.603+0000 7f80b5979700 1 -- 172.21.15.22:0/3134653546 reap_dead start 2023-07-06T01:20:34.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.603+0000 7f80b73dc700 1 -- 172.21.15.22:0/3134653546 shutdown_connections 2023-07-06T01:20:34.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.603+0000 7f80b73dc700 1 -- 172.21.15.22:0/3134653546 >> 172.21.15.22:0/3134653546 conn(0x7f80b00999f0 msgr2=0x7f80b0099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:34.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.603+0000 7f80b73dc700 1 -- 172.21.15.22:0/3134653546 shutdown_connections 2023-07-06T01:20:34.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.603+0000 7f80b73dc700 1 -- 172.21.15.22:0/3134653546 wait complete. 2023-07-06T01:20:34.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.604+0000 7f80b73dc700 1 Processor -- start 2023-07-06T01:20:34.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.604+0000 7f80b73dc700 1 -- start start 2023-07-06T01:20:34.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b73dc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f809c004d90 0x7f80b00b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b73dc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80b00b3c00 0x7f80b01588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.612 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b73dc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b01ef610 0x7f80b00ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b73dc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00b56e0 con 0x7f80b01ef610 2023-07-06T01:20:34.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b73dc700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00b4100 con 0x7f809c004d90 2023-07-06T01:20:34.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b73dc700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00b43e0 con 0x7f80b00b3c00 2023-07-06T01:20:34.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b5979700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b01ef610 0x7f80b00ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b5979700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b01ef610 0x7f80b00ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47536/0 (socket says 172.21.15.22:47536) 2023-07-06T01:20:34.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b5979700 1 -- 172.21.15.22:0/560608297 learned_addr learned my addr 172.21.15.22:0/560608297 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:34.613 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b5178700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f809c004d90 0x7f80b00b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b4977700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80b00b3c00 0x7f80b01588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b5979700 1 -- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80b00b3c00 msgr2=0x7f80b01588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:34.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b5979700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80b00b3c00 0x7f80b01588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b5979700 1 -- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f809c004d90 msgr2=0x7f80b00b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:34.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b5979700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f809c004d90 0x7f80b00b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b5979700 1 -- 172.21.15.22:0/560608297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b00b46c0 con 0x7f80b01ef610 2023-07-06T01:20:34.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b5979700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b01ef610 0x7f80b00ca1d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f80a8017040 tx=0x7f80a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:34.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80a1ffb700 1 -- 172.21.15.22:0/560608297 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8007000 con 0x7f80b01ef610 2023-07-06T01:20:34.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80a1ffb700 1 -- 172.21.15.22:0/560608297 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a8003e50 con 0x7f80b01ef610 2023-07-06T01:20:34.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80b73dc700 1 -- 172.21.15.22:0/560608297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b00b49a0 con 0x7f80b01ef610 2023-07-06T01:20:34.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.605+0000 7f80a1ffb700 1 -- 172.21.15.22:0/560608297 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8014aa0 con 0x7f80b01ef610 2023-07-06T01:20:34.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.606+0000 7f80b73dc700 1 -- 172.21.15.22:0/560608297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b0158e00 con 0x7f80b01ef610 2023-07-06T01:20:34.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.607+0000 7f80b73dc700 1 -- 172.21.15.22:0/560608297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b00b4ec0 con 0x7f80b01ef610 2023-07-06T01:20:34.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.607+0000 7f80a1ffb700 1 -- 172.21.15.22:0/560608297 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f80a8023020 con 0x7f80b01ef610 2023-07-06T01:20:34.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.607+0000 7f80a1ffb700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f809406ace0 0x7f809406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:34.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.608+0000 7f80a1ffb700 1 -- 172.21.15.22:0/560608297 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f80a808a9a0 con 0x7f80b01ef610 2023-07-06T01:20:34.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.608+0000 7f80b5178700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f809406ace0 0x7f809406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:34.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.608+0000 7f80b5178700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f809406ace0 0x7f809406d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f80b0157750 tx=0x7f80ac009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:34.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.612+0000 7f80a1ffb700 1 -- 172.21.15.22:0/560608297 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f80b00b4ec0 con 0x7f80b01ef610 2023-07-06T01:20:34.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.856+0000 7f80b73dc700 1 -- 172.21.15.22:0/560608297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80b00b4ec0 con 0x7f80b01ef610 2023-07-06T01:20:34.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.856+0000 7f80a1ffb700 1 -- 172.21.15.22:0/560608297 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4671 (secure 0 0 0) 0x7f80b00b4ec0 con 0x7f80b01ef610 2023-07-06T01:20:34.858 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:20:34.859 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/219695916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":219695916},{"type":"v1","addr":"172.21.15.103:6836","nonce":219695916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3102932229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":3102932229},{"type":"v1","addr":"172.21.15.103:6837","nonce":3102932229}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3787686782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3787686782},{"type":"v1","addr":"172.21.15.22:6834","nonce":3787686782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T01:20:24.257974+0000","modified":"2023-07-06T01:20:27.865213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6835/612827809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":612827809},{"type":"v1","addr":"172.21.15.22:6835","nonce":612827809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T01:20:34.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.858+0000 7f80b73dc700 1 -- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f809406ace0 msgr2=0x7f809406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:34.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.858+0000 7f80b73dc700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f809406ace0 0x7f809406d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f80b0157750 tx=0x7f80ac009210 comp rx=0 tx=0).stop 2023-07-06T01:20:34.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.858+0000 7f80b73dc700 1 -- 172.21.15.22:0/560608297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b01ef610 msgr2=0x7f80b00ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:34.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.858+0000 7f80b73dc700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b01ef610 0x7f80b00ca1d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f80a8017040 tx=0x7f80a8009750 comp rx=0 tx=0).stop 2023-07-06T01:20:34.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.858+0000 7f80b73dc700 1 -- 172.21.15.22:0/560608297 shutdown_connections 2023-07-06T01:20:34.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.858+0000 7f80b73dc700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80b01ef610 0x7f80b00ca1d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.858+0000 7f80b73dc700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f809406ace0 0x7f809406d190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.858+0000 7f80b73dc700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f809c004d90 0x7f80b00b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.858+0000 7f80b73dc700 1 --2- 172.21.15.22:0/560608297 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80b00b3c00 0x7f80b01588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:34.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.858+0000 7f80b73dc700 1 -- 172.21.15.22:0/560608297 >> 172.21.15.22:0/560608297 conn(0x7f80b00999f0 msgr2=0x7f80b006dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:34.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.859+0000 7f80b73dc700 1 -- 172.21.15.22:0/560608297 shutdown_connections 2023-07-06T01:20:34.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:34.859+0000 7f80b73dc700 1 -- 172.21.15.22:0/560608297 wait complete. 2023-07-06T01:20:34.862 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 15 2023-07-06T01:20:35.275 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T01:20:24.257974+0000', 'modified': '2023-07-06T01:20:27.865213+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5115': {'gid': 5115, 'name': 'c', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.22:6835/612827809', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6833', 'nonce': 612827809}, {'type': 'v1', 'addr': '172.21.15.22:6835', 'nonce': 612827809}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T01:20:35.275 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T01:20:35.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-07-06T01:20:35.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.404+0000 7f1f90aaf700 1 Processor -- start 2023-07-06T01:20:35.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.405+0000 7f1f90aaf700 1 -- start start 2023-07-06T01:20:35.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.405+0000 7f1f90aaf700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f8c0ccd70 0x7f1f8c15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.405+0000 7f1f90aaf700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f1f8c06d040 0x7f1f8c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.405+0000 7f1f90aaf700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8c0cf230 con 0x7f1f8c159360 2023-07-06T01:20:35.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.405+0000 7f1f90aaf700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8c15df20 con 0x7f1f8c06d040 2023-07-06T01:20:35.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.405+0000 7f1f90aaf700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0ce620 con 0x7f1f8c0ccd70 2023-07-06T01:20:35.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.406+0000 7f1f8ad9d700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f1f8c06d040 0x7f1f8c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.406+0000 7f1f8ad9d700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f1f8c06d040 0x7f1f8c06d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:36268/0 (socket says 172.21.15.22:36268) 2023-07-06T01:20:35.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.406+0000 7f1f8ad9d700 1 -- 172.21.15.22:0/735015956 learned_addr learned my addr 172.21.15.22:0/735015956 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:35.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.406+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/735015956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f8c0ccd70 0x7f1f8c15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.406+0000 7f1f89d9b700 1 -- 172.21.15.22:0/735015956 >> v2:172.21.15.103:3300/0 conn(0x7f1f8c06d040 msgr2=0x7f1f8c06d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.406+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/735015956 >> v2:172.21.15.103:3300/0 conn(0x7f1f8c06d040 0x7f1f8c06d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.406+0000 7f1f89d9b700 1 -- 172.21.15.22:0/735015956 >> v1:172.21.15.103:6789/0 conn(0x7f1f8c159360 legacy=0x7f1f8c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.406+0000 7f1f89d9b700 1 -- 172.21.15.22:0/735015956 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c0ce900 con 0x7f1f8c0ccd70 2023-07-06T01:20:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.406+0000 7f1f8ad9d700 1 --2- 172.21.15.22:0/735015956 >> v2:172.21.15.103:3300/0 conn(0x7f1f8c06d040 0x7f1f8c06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.407+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/735015956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f8c0ccd70 0x7f1f8c15d9e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009840 tx=0x7f1f74009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a15cd58b2c3f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:35.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.407+0000 7f1f8959a700 1 -- 172.21.15.22:0/735015956 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74011830 con 0x7f1f8c0ccd70 2023-07-06T01:20:35.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.407+0000 7f1f8959a700 1 -- 172.21.15.22:0/735015956 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f74011990 con 0x7f1f8c0ccd70 2023-07-06T01:20:35.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.407+0000 7f1f8959a700 1 -- 172.21.15.22:0/735015956 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74011d20 con 0x7f1f8c0ccd70 2023-07-06T01:20:35.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.407+0000 7f1f90aaf700 1 -- 172.21.15.22:0/735015956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f8c0ccd70 msgr2=0x7f1f8c15d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.407+0000 7f1f90aaf700 1 --2- 172.21.15.22:0/735015956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f8c0ccd70 0x7f1f8c15d9e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009840 tx=0x7f1f74009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:35.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.407+0000 7f1f90aaf700 1 -- 172.21.15.22:0/735015956 shutdown_connections 2023-07-06T01:20:35.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.407+0000 7f1f90aaf700 1 --2- 172.21.15.22:0/735015956 >> v2:172.21.15.103:3300/0 conn(0x7f1f8c06d040 0x7f1f8c06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.407+0000 7f1f90aaf700 1 --2- 172.21.15.22:0/735015956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f8c0ccd70 0x7f1f8c15d9e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.407+0000 7f1f90aaf700 1 -- 172.21.15.22:0/735015956 >> 172.21.15.22:0/735015956 conn(0x7f1f8c0b1890 msgr2=0x7f1f8c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:35.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.407+0000 7f1f90aaf700 1 -- 172.21.15.22:0/735015956 shutdown_connections 2023-07-06T01:20:35.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.407+0000 7f1f90aaf700 1 -- 172.21.15.22:0/735015956 wait complete. 2023-07-06T01:20:35.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.409+0000 7f1f90aaf700 1 Processor -- start 2023-07-06T01:20:35.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.409+0000 7f1f90aaf700 1 -- start start 2023-07-06T01:20:35.414 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.409+0000 7f1f90aaf700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f8c06d040 0x7f1f8c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.409+0000 7f1f90aaf700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f8c0ccd70 0x7f1f8c15b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.409+0000 7f1f90aaf700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f8c159360 0x7f1f8c0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.409+0000 7f1f90aaf700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0c8880 con 0x7f1f8c06d040 2023-07-06T01:20:35.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.409+0000 7f1f90aaf700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0c7f50 con 0x7f1f8c0ccd70 2023-07-06T01:20:35.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.409+0000 7f1f90aaf700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0c8230 con 0x7f1f8c159360 2023-07-06T01:20:35.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f8a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f8c06d040 0x7f1f8c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.415 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f8a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f8c06d040 0x7f1f8c0b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47546/0 (socket says 172.21.15.22:47546) 2023-07-06T01:20:35.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f8a59c700 1 -- 172.21.15.22:0/1294103420 learned_addr learned my addr 172.21.15.22:0/1294103420 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:35.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f8c0ccd70 0x7f1f8c15b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f8ad9d700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f8c159360 0x7f1f8c0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.416 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f8a59c700 1 -- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f8c159360 msgr2=0x7f1f8c0b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f8a59c700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f8c159360 0x7f1f8c0b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f8a59c700 1 -- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f8c0ccd70 msgr2=0x7f1f8c15b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f8a59c700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f8c0ccd70 0x7f1f8c15b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f8a59c700 1 -- 172.21.15.22:0/1294103420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f84009b30 con 0x7f1f8c06d040 2023-07-06T01:20:35.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f8a59c700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f8c06d040 0x7f1f8c0b2b90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00c430 tx=0x7f1f7c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:35.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f82ffd700 1 -- 172.21.15.22:0/1294103420 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0050d0 con 0x7f1f8c06d040 2023-07-06T01:20:35.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f82ffd700 1 -- 172.21.15.22:0/1294103420 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c005250 con 0x7f1f8c06d040 2023-07-06T01:20:35.417 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f90aaf700 1 -- 172.21.15.22:0/1294103420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f740094a0 con 0x7f1f8c06d040 2023-07-06T01:20:35.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f82ffd700 1 -- 172.21.15.22:0/1294103420 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c006860 con 0x7f1f8c06d040 2023-07-06T01:20:35.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.410+0000 7f1f90aaf700 1 -- 172.21.15.22:0/1294103420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c15c160 con 0x7f1f8c06d040 2023-07-06T01:20:35.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.412+0000 7f1f82ffd700 1 -- 172.21.15.22:0/1294103420 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f7c003680 con 0x7f1f8c06d040 2023-07-06T01:20:35.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.412+0000 7f1f82ffd700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1f6c06ace0 0x7f1f6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.412+0000 7f1f82ffd700 1 -- 172.21.15.22:0/1294103420 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f7c088650 con 0x7f1f8c06d040 2023-07-06T01:20:35.418 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.413+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1f6c06ace0 0x7f1f6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.413+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1f6c06ace0 0x7f1f6c06d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1f74000c00 tx=0x7f1f74008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.414+0000 7f1f9cc34b80 1 Processor -- start 2023-07-06T01:20:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.414+0000 7f1f9cc34b80 1 -- start start 2023-07-06T01:20:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.415+0000 7f1f9cc34b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 0x5565ba94a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.415+0000 7f1f9cc34b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 0x5565ba92e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.415+0000 7f1f9cc34b80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba92bcd0 0x5565ba92e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.415+0000 7f1f9cc34b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba808c60 con 0x5565ba901830 2023-07-06T01:20:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.415+0000 7f1f9cc34b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba85b9f0 con 0x5565ba92b470 2023-07-06T01:20:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.415+0000 7f1f9cc34b80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565ba854e80 con 0x5565ba92bcd0 2023-07-06T01:20:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.415+0000 7f1f89d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 0x5565ba92e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.415+0000 7f1f89d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 0x5565ba92e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47556/0 (socket says 172.21.15.22:47556) 2023-07-06T01:20:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.415+0000 7f1f89d9b700 1 -- 172.21.15.22:0/3324585210 learned_addr learned my addr 172.21.15.22:0/3324585210 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.415+0000 7f1f8a59c700 1 --2- 172.21.15.22:0/3324585210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 0x5565ba94a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.415+0000 7f1f8ad9d700 1 --2- 172.21.15.22:0/3324585210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba92bcd0 0x5565ba92e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.416+0000 7f1f89d9b700 1 -- 172.21.15.22:0/3324585210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba92bcd0 msgr2=0x5565ba92e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.416+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/3324585210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba92bcd0 0x5565ba92e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.416+0000 7f1f89d9b700 1 -- 172.21.15.22:0/3324585210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 msgr2=0x5565ba94a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.416+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/3324585210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 0x5565ba94a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.416+0000 7f1f89d9b700 1 -- 172.21.15.22:0/3324585210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565ba682d90 con 0x5565ba901830 2023-07-06T01:20:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.416+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/3324585210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 0x5565ba92e1f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c15e060 tx=0x7f1f74010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec7618e2c8fe8fb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.416+0000 7f1f6bfff700 1 -- 172.21.15.22:0/3324585210 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7402c070 con 0x5565ba901830 2023-07-06T01:20:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.416+0000 7f1f6bfff700 1 -- 172.21.15.22:0/3324585210 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f740036a0 con 0x5565ba901830 2023-07-06T01:20:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.416+0000 7f1f6bfff700 1 -- 172.21.15.22:0/3324585210 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74020070 con 0x5565ba901830 2023-07-06T01:20:35.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.417+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3324585210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 msgr2=0x5565ba92e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.417+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3324585210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 0x5565ba92e1f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c15e060 tx=0x7f1f74010000 comp rx=0 tx=0).stop 2023-07-06T01:20:35.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.417+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3324585210 shutdown_connections 2023-07-06T01:20:35.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.417+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3324585210 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba92bcd0 0x5565ba92e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.417+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3324585210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 0x5565ba94a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.417+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3324585210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 0x5565ba92e1f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.417+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3324585210 >> 172.21.15.22:0/3324585210 conn(0x5565ba8f8e80 msgr2=0x5565ba9491f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:35.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.417+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3324585210 shutdown_connections 2023-07-06T01:20:35.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.417+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3324585210 wait complete. 2023-07-06T01:20:35.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.417+0000 7f1f9cc34b80 1 -- start start 2023-07-06T01:20:35.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.417+0000 7f1f9cc34b80 1 Processor -- start 2023-07-06T01:20:35.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f9cc34b80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba901830 0x5565ba940b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f9cc34b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92b470 0x5565ba94c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f9cc34b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92bcd0 0x5565ba94ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f9cc34b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba7f4a70 con 0x5565ba92b470 2023-07-06T01:20:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f9cc34b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba7f4770 con 0x5565ba92bcd0 2023-07-06T01:20:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f9cc34b80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565ba6d88d0 con 0x5565ba901830 2023-07-06T01:20:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f89d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92b470 0x5565ba94c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f9cc34b80 10 client.0 _ll_register_callbacks cb 0x5565ba7f2710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T01:20:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f89d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92b470 0x5565ba94c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47562/0 (socket says 172.21.15.22:47562) 2023-07-06T01:20:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f89d9b700 1 -- 172.21.15.22:0/1454291074 learned_addr learned my addr 172.21.15.22:0/1454291074 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f8a59c700 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba901830 0x5565ba940b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f8ad9d700 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92bcd0 0x5565ba94ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f89d9b700 1 -- 172.21.15.22:0/1454291074 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba901830 msgr2=0x5565ba940b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba901830 0x5565ba940b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f89d9b700 1 -- 172.21.15.22:0/1454291074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92bcd0 msgr2=0x5565ba94ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92bcd0 0x5565ba94ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f89d9b700 1 -- 172.21.15.22:0/1454291074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c0ce900 con 0x5565ba92b470 2023-07-06T01:20:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f8ad9d700 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92bcd0 0x5565ba94ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:35.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.419+0000 7f1f8a59c700 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba901830 0x5565ba940b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:20:35.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92b470 0x5565ba94c510 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1f74033040 tx=0x7f1f740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde26d03813fc199 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:35.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f697fa700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-07-06T01:20:35.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f697fa700 1 -- 172.21.15.22:0/1454291074 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74003de0 con 0x5565ba92b470 2023-07-06T01:20:35.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f697fa700 1 -- 172.21.15.22:0/1454291074 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f74026410 con 0x5565ba92b470 2023-07-06T01:20:35.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/1454291074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565ba8f6490 con 0x5565ba92b470 2023-07-06T01:20:35.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f697fa700 1 -- 172.21.15.22:0/1454291074 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f740265d0 con 0x5565ba92b470 2023-07-06T01:20:35.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f9cc34b80 10 client.5310 fetch_fsmap learned FSMap version 15 2023-07-06T01:20:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/1454291074 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5565ba66bc10 con 0x5565ba92b470 2023-07-06T01:20:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f697fa700 1 -- 172.21.15.22:0/1454291074 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5565ba8f6490 con 0x5565ba92b470 2023-07-06T01:20:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f697fa700 1 -- 172.21.15.22:0/1454291074 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1f74026730 con 0x5565ba92b470 2023-07-06T01:20:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f9cc34b80 10 client.5310 fetch_fsmap finished waiting for FSMap version 15 2023-07-06T01:20:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f9cc34b80 10 client.5310 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T01:20:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f9cc34b80 20 client.5310 populate_metadata read hostname 'smithi022' 2023-07-06T01:20:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5565ba927c00 0x5565ba927fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f9cc34b80 4 client.5310 mds_command: new command op to 5115 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T01:20:35.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/1454291074 --> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565ba84de60 con 0x5565ba927c00 2023-07-06T01:20:35.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.420+0000 7f1f8a59c700 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5565ba927c00 0x5565ba927fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.421+0000 7f1f697fa700 10 client.5310 ms_handle_connect on v2:172.21.15.22:6833/612827809 2023-07-06T01:20:35.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.421+0000 7f1f8a59c700 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5565ba927c00 0x5565ba927fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=488f92b4719416b4 server_cookie=51a1056cfe54363 in_seq=0 out_seq=0 2023-07-06T01:20:35.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.421+0000 7f1f697fa700 1 -- 172.21.15.22:0/1454291074 <== mds.0 v2:172.21.15.22:6833/612827809 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5565ba84de60 con 0x5565ba927c00 2023-07-06T01:20:35.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.421+0000 7f1f697fa700 10 client.5310 handle_command_reply: tid=0 2023-07-06T01:20:35.433 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.421+0000 7f1f9cc34b80 1 client.5310 shutdown 2023-07-06T01:20:35.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.421+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/1454291074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92b470 msgr2=0x5565ba94c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.421+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92b470 0x5565ba94c510 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1f74033040 tx=0x7f1f740047c0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.422+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/1454291074 shutdown_connections 2023-07-06T01:20:35.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.422+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92bcd0 0x5565ba94ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.422+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba901830 0x5565ba940b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.422+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5565ba927c00 0x5565ba927fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.422+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/1454291074 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92b470 0x5565ba94c510 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.422+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/1454291074 >> 172.21.15.22:0/1454291074 conn(0x5565ba8f8e80 msgr2=0x5565ba929230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.422+0000 7f1f697fa700 0 client.5310 ms_handle_reset on v2:172.21.15.22:6833/612827809 2023-07-06T01:20:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.422+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/1454291074 shutdown_connections 2023-07-06T01:20:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.422+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/1454291074 wait complete. 2023-07-06T01:20:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.422+0000 7f1f9cc34b80 20 client.5310 trim_cache size 0 max 16384 2023-07-06T01:20:35.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.440+0000 7f1f9cc34b80 1 Processor -- start 2023-07-06T01:20:35.441 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.440+0000 7f1f9cc34b80 1 -- start start 2023-07-06T01:20:35.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.440+0000 7f1f9cc34b80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba901830 0x5565ba96b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.440+0000 7f1f9cc34b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 0x5565ba904770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.440+0000 7f1f9cc34b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92bcd0 0x5565ba949280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.440+0000 7f1f9cc34b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba929330 con 0x5565ba92bcd0 2023-07-06T01:20:35.443 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.440+0000 7f1f9cc34b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba959100 con 0x5565ba92b470 2023-07-06T01:20:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f9cc34b80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565ba928cc0 con 0x5565ba901830 2023-07-06T01:20:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f8ad9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92bcd0 0x5565ba949280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f8ad9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92bcd0 0x5565ba949280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47570/0 (socket says 172.21.15.22:47570) 2023-07-06T01:20:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f8ad9d700 1 -- 172.21.15.22:0/3069645848 learned_addr learned my addr 172.21.15.22:0/3069645848 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/3069645848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 0x5565ba904770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f8a59c700 1 --2- 172.21.15.22:0/3069645848 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba901830 0x5565ba96b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f8ad9d700 1 -- 172.21.15.22:0/3069645848 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba901830 msgr2=0x5565ba96b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f8ad9d700 1 --2- 172.21.15.22:0/3069645848 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba901830 0x5565ba96b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f8ad9d700 1 -- 172.21.15.22:0/3069645848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 msgr2=0x5565ba904770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f8ad9d700 1 --2- 172.21.15.22:0/3069645848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 0x5565ba904770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f8ad9d700 1 -- 172.21.15.22:0/3069645848 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565ba6a9200 con 0x5565ba92bcd0 2023-07-06T01:20:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f8ad9d700 1 --2- 172.21.15.22:0/3069645848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92bcd0 0x5565ba949280 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1f84007810 tx=0x7f1f84009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5951d18f2fe46433 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f80ff9700 1 -- 172.21.15.22:0/3069645848 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8400e040 con 0x5565ba92bcd0 2023-07-06T01:20:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f80ff9700 1 -- 172.21.15.22:0/3069645848 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8400f040 con 0x5565ba92bcd0 2023-07-06T01:20:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.441+0000 7f1f80ff9700 1 -- 172.21.15.22:0/3069645848 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84004270 con 0x5565ba92bcd0 2023-07-06T01:20:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.442+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3069645848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92bcd0 msgr2=0x5565ba949280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.442+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3069645848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92bcd0 0x5565ba949280 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1f84007810 tx=0x7f1f84009810 comp rx=0 tx=0).stop 2023-07-06T01:20:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.442+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3069645848 shutdown_connections 2023-07-06T01:20:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.442+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3069645848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba92bcd0 0x5565ba949280 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.442+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3069645848 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba901830 0x5565ba96b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.442+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3069645848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 0x5565ba904770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.442+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3069645848 >> 172.21.15.22:0/3069645848 conn(0x5565ba8f8e80 msgr2=0x5565ba957590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.442+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3069645848 shutdown_connections 2023-07-06T01:20:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.442+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3069645848 wait complete. 2023-07-06T01:20:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.442+0000 7f1f9cc34b80 1 -- start start 2023-07-06T01:20:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.442+0000 7f1f9cc34b80 1 Processor -- start 2023-07-06T01:20:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.443+0000 7f1f9cc34b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 0x5565ba929b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.443+0000 7f1f9cc34b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 0x5565ba92a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.443+0000 7f1f9cc34b80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba92bcd0 0x5565ba9490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.443+0000 7f1f9cc34b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba9968b0 con 0x5565ba901830 2023-07-06T01:20:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.443+0000 7f1f9cc34b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ba92a690 con 0x5565ba92b470 2023-07-06T01:20:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.443+0000 7f1f9cc34b80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565ba999450 con 0x5565ba92bcd0 2023-07-06T01:20:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.443+0000 7f1f9cc34b80 10 client.0 _ll_register_callbacks cb 0x7f1f7c053670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T01:20:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.443+0000 7f1f8a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 0x5565ba929b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f8a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 0x5565ba929b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47574/0 (socket says 172.21.15.22:47574) 2023-07-06T01:20:35.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f8a59c700 1 -- 172.21.15.22:0/3631453805 learned_addr learned my addr 172.21.15.22:0/3631453805 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:35.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f8ad9d700 1 --2- 172.21.15.22:0/3631453805 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba92bcd0 0x5565ba9490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/3631453805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 0x5565ba92a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f8a59c700 1 -- 172.21.15.22:0/3631453805 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba92bcd0 msgr2=0x5565ba9490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:35.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f8a59c700 1 --2- 172.21.15.22:0/3631453805 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba92bcd0 0x5565ba9490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f8a59c700 1 -- 172.21.15.22:0/3631453805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 msgr2=0x5565ba92a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:35.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f8a59c700 1 --2- 172.21.15.22:0/3631453805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 0x5565ba92a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f8a59c700 1 -- 172.21.15.22:0/3631453805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f840074f0 con 0x5565ba901830 2023-07-06T01:20:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f8a59c700 1 --2- 172.21.15.22:0/3631453805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 0x5565ba929b10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c057030 tx=0x7f1f7c0868f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e944526563ed8340 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f697fa700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-07-06T01:20:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f697fa700 1 -- 172.21.15.22:0/3631453805 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c05f480 con 0x5565ba901830 2023-07-06T01:20:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f697fa700 1 -- 172.21.15.22:0/3631453805 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f7c086e40 con 0x5565ba901830 2023-07-06T01:20:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3631453805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f740036a0 con 0x5565ba901830 2023-07-06T01:20:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f697fa700 1 -- 172.21.15.22:0/3631453805 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f7c055530 con 0x5565ba901830 2023-07-06T01:20:35.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f9cc34b80 10 client.5316 fetch_fsmap learned FSMap version 15 2023-07-06T01:20:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3631453805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5565ba681260 con 0x5565ba901830 2023-07-06T01:20:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.444+0000 7f1f697fa700 1 -- 172.21.15.22:0/3631453805 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c078070 con 0x5565ba901830 2023-07-06T01:20:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f697fa700 1 -- 172.21.15.22:0/3631453805 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1f7c080b60 con 0x5565ba901830 2023-07-06T01:20:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 10 client.5316 fetch_fsmap finished waiting for FSMap version 15 2023-07-06T01:20:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 10 client.5316 resolve_mds: resolved name 'c' to daemon mds.c 2023-07-06T01:20:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 20 client.5316 populate_metadata read hostname 'smithi022' 2023-07-06T01:20:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3631453805 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5565ba9021a0 0x5565ba902570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 4 client.5316 mds_command: new command op to 5115 tid=0[{"prefix": "status"}] 2023-07-06T01:20:35.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3631453805 --> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5565ba810000 con 0x5565ba9021a0 2023-07-06T01:20:35.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/3631453805 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5565ba9021a0 0x5565ba902570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f697fa700 10 client.5316 ms_handle_connect on v2:172.21.15.22:6833/612827809 2023-07-06T01:20:35.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f89d9b700 1 --2- 172.21.15.22:0/3631453805 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5565ba9021a0 0x5565ba902570 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6b924944f3ffd2b server_cookie=681e2c240950e173 in_seq=0 out_seq=0 2023-07-06T01:20:35.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f697fa700 1 -- 172.21.15.22:0/3631453805 <== mds.0 v2:172.21.15.22:6833/612827809 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5565ba810000 con 0x5565ba9021a0 2023-07-06T01:20:35.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f697fa700 10 client.5316 handle_command_reply: tid=0 2023-07-06T01:20:35.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 1 client.5316 shutdown 2023-07-06T01:20:35.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3631453805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 msgr2=0x5565ba929b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3631453805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 0x5565ba929b10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c057030 tx=0x7f1f7c0868f0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3631453805 shutdown_connections 2023-07-06T01:20:35.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3631453805 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5565ba92bcd0 0x5565ba9490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3631453805 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5565ba9021a0 0x5565ba902570 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3631453805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5565ba901830 0x5565ba929b10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f697fa700 0 client.5316 ms_handle_reset on v2:172.21.15.22:6833/612827809 2023-07-06T01:20:35.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 1 --2- 172.21.15.22:0/3631453805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5565ba92b470 0x5565ba92a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.445+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3631453805 >> 172.21.15.22:0/3631453805 conn(0x5565ba8f8e80 msgr2=0x5565ba927fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3631453805 shutdown_connections 2023-07-06T01:20:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f9cc34b80 1 -- 172.21.15.22:0/3631453805 wait complete. 2023-07-06T01:20:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f9cc34b80 20 client.5316 trim_cache size 0 max 16384 2023-07-06T01:20:35.457 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:20:35.457 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "63a894dd-9157-4ce0-adf3-c1bdfe16ec64", 2023-07-06T01:20:35.458 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2023-07-06T01:20:35.458 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 5115, 2023-07-06T01:20:35.458 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2023-07-06T01:20:35.458 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2023-07-06T01:20:35.458 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2023-07-06T01:20:35.458 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 10.61429457, 2023-07-06T01:20:35.458 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 15, 2023-07-06T01:20:35.458 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 34, 2023-07-06T01:20:35.459 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 34, 2023-07-06T01:20:35.459 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 19.566463607999999 2023-07-06T01:20:35.459 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:20:35.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f90aaf700 1 -- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1f6c06ace0 msgr2=0x7f1f6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f90aaf700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1f6c06ace0 0x7f1f6c06d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1f74000c00 tx=0x7f1f74008030 comp rx=0 tx=0).stop 2023-07-06T01:20:35.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f90aaf700 1 -- 172.21.15.22:0/1294103420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f8c06d040 msgr2=0x7f1f8c0b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f90aaf700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f8c06d040 0x7f1f8c0b2b90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00c430 tx=0x7f1f7c00c740 comp rx=0 tx=0).stop 2023-07-06T01:20:35.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f90aaf700 1 -- 172.21.15.22:0/1294103420 shutdown_connections 2023-07-06T01:20:35.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f90aaf700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f8c159360 0x7f1f8c0b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f90aaf700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f8c06d040 0x7f1f8c0b2b90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f90aaf700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1f6c06ace0 0x7f1f6c06d190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f90aaf700 1 --2- 172.21.15.22:0/1294103420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f8c0ccd70 0x7f1f8c15b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f90aaf700 1 -- 172.21.15.22:0/1294103420 >> 172.21.15.22:0/1294103420 conn(0x7f1f8c0b1890 msgr2=0x7f1f8c15cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:35.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f90aaf700 1 -- 172.21.15.22:0/1294103420 shutdown_connections 2023-07-06T01:20:35.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.446+0000 7f1f90aaf700 1 -- 172.21.15.22:0/1294103420 wait complete. 2023-07-06T01:20:35.462 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T01:20:35.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.585+0000 7f6209abe700 1 Processor -- start 2023-07-06T01:20:35.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.585+0000 7f6209abe700 1 -- start start 2023-07-06T01:20:35.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.586+0000 7f6209abe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f62040b3b70 0x7f62040d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.586+0000 7f6209abe700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62040cebd0 con 0x7f62040b40c0 2023-07-06T01:20:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.586+0000 7f6209abe700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62040cdfc0 con 0x7f62040b36f0 2023-07-06T01:20:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.586+0000 7f6209abe700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620415cff0 con 0x7f62040b3b70 2023-07-06T01:20:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.586+0000 7f6202ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f62040b3b70 0x7f62040d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.586+0000 7f6203fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f62040b40c0 0x7f62040b5da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38330/0 (socket says 172.21.15.22:38330) 2023-07-06T01:20:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.586+0000 7f6203fff700 1 -- 172.21.15.22:0/3957638080 learned_addr learned my addr 172.21.15.22:0/3957638080 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.587+0000 7f6202ffd700 1 -- 172.21.15.22:0/3957638080 >> v1:172.21.15.103:6789/0 conn(0x7f62040b36f0 legacy=0x7f62040cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.587+0000 7f6202ffd700 1 -- 172.21.15.22:0/3957638080 >> v1:172.21.15.22:6789/0 conn(0x7f62040b40c0 legacy=0x7f62040b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.587+0000 7f62027fc700 1 -- 172.21.15.22:0/3957638080 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379003447 0 0) 0x7f62040cebd0 con 0x7f62040b40c0 2023-07-06T01:20:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.587+0000 7f6202ffd700 1 -- 172.21.15.22:0/3957638080 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62040ce2a0 con 0x7f62040b3b70 2023-07-06T01:20:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.587+0000 7f6202ffd700 1 --2- 172.21.15.22:0/3957638080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040b3b70 0x7f62040d0a90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009840 tx=0x7f61f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3cffff1ea3db2e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.587+0000 7f62027fc700 1 -- 172.21.15.22:0/3957638080 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8011830 con 0x7f62040b3b70 2023-07-06T01:20:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.587+0000 7f62027fc700 1 -- 172.21.15.22:0/3957638080 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f8011990 con 0x7f62040b3b70 2023-07-06T01:20:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.587+0000 7f62027fc700 1 -- 172.21.15.22:0/3957638080 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8011d20 con 0x7f62040b3b70 2023-07-06T01:20:35.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.587+0000 7f6209abe700 1 -- 172.21.15.22:0/3957638080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040b3b70 msgr2=0x7f62040d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.587+0000 7f6209abe700 1 --2- 172.21.15.22:0/3957638080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040b3b70 0x7f62040d0a90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009840 tx=0x7f61f8009b50 comp rx=0 tx=0).stop 2023-07-06T01:20:35.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.587+0000 7f6209abe700 1 -- 172.21.15.22:0/3957638080 shutdown_connections 2023-07-06T01:20:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.587+0000 7f6209abe700 1 --2- 172.21.15.22:0/3957638080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040b3b70 0x7f62040d0a90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.587+0000 7f6209abe700 1 -- 172.21.15.22:0/3957638080 >> 172.21.15.22:0/3957638080 conn(0x7f620409c2e0 msgr2=0x7f62040d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.588+0000 7f6209abe700 1 -- 172.21.15.22:0/3957638080 shutdown_connections 2023-07-06T01:20:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.588+0000 7f6209abe700 1 -- 172.21.15.22:0/3957638080 wait complete. 2023-07-06T01:20:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.589+0000 7f6209abe700 1 Processor -- start 2023-07-06T01:20:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.589+0000 7f6209abe700 1 -- start start 2023-07-06T01:20:35.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.590+0000 7f6209abe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f62040b36f0 0x7f62040ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.590+0000 7f6209abe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62040b3b70 0x7f62040cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.594 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.590+0000 7f6209abe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040b40c0 0x7f62040b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.590+0000 7f6209abe700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62040c8450 con 0x7f62040b40c0 2023-07-06T01:20:35.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.590+0000 7f6209abe700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62040c8710 con 0x7f62040b3b70 2023-07-06T01:20:35.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.590+0000 7f6209abe700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62040ccea0 con 0x7f62040b36f0 2023-07-06T01:20:35.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.590+0000 7f6203fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040b40c0 0x7f62040b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.590+0000 7f6203fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040b40c0 0x7f62040b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47592/0 (socket says 172.21.15.22:47592) 2023-07-06T01:20:35.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.590+0000 7f6203fff700 1 -- 172.21.15.22:0/1127192935 learned_addr learned my addr 172.21.15.22:0/1127192935 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:35.595 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.590+0000 7f62037fe700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f62040b36f0 0x7f62040ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.590+0000 7f6202ffd700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62040b3b70 0x7f62040cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.591+0000 7f6203fff700 1 -- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f62040b36f0 msgr2=0x7f62040ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.591+0000 7f6203fff700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f62040b36f0 0x7f62040ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.591+0000 7f6203fff700 1 -- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62040b3b70 msgr2=0x7f62040cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.591+0000 7f6203fff700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62040b3b70 0x7f62040cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.591+0000 7f6203fff700 1 -- 172.21.15.22:0/1127192935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f80094a0 con 0x7f62040b40c0 2023-07-06T01:20:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.591+0000 7f6203fff700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040b40c0 0x7f62040b6dd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00cc70 tx=0x7f61fc00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.591+0000 7f61ebfff700 1 -- 172.21.15.22:0/1127192935 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc00aac0 con 0x7f62040b40c0 2023-07-06T01:20:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.591+0000 7f61ebfff700 1 -- 172.21.15.22:0/1127192935 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61fc00ac40 con 0x7f62040b40c0 2023-07-06T01:20:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.591+0000 7f6209abe700 1 -- 172.21.15.22:0/1127192935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62040ca7f0 con 0x7f62040b40c0 2023-07-06T01:20:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.591+0000 7f61ebfff700 1 -- 172.21.15.22:0/1127192935 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0070d0 con 0x7f62040b40c0 2023-07-06T01:20:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.591+0000 7f6209abe700 1 -- 172.21.15.22:0/1127192935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62040cb650 con 0x7f62040b40c0 2023-07-06T01:20:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.592+0000 7f6209abe700 1 -- 172.21.15.22:0/1127192935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62040b4840 con 0x7f62040b40c0 2023-07-06T01:20:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.593+0000 7f61ebfff700 1 -- 172.21.15.22:0/1127192935 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f61fc007250 con 0x7f62040b40c0 2023-07-06T01:20:35.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.593+0000 7f61ebfff700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f61f006ace0 0x7f61f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.593+0000 7f61ebfff700 1 -- 172.21.15.22:0/1127192935 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61fc0891d0 con 0x7f62040b40c0 2023-07-06T01:20:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.593+0000 7f62037fe700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f61f006ace0 0x7f61f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.594+0000 7f62037fe700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f61f006ace0 0x7f61f006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f61f4007fd0 tx=0x7f61f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.597+0000 7f61ebfff700 1 -- 172.21.15.22:0/1127192935 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f61fc054310 con 0x7f62040b40c0 2023-07-06T01:20:35.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.836+0000 7f6209abe700 1 -- 172.21.15.22:0/1127192935 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f62040c88c0 con 0x7f62040b40c0 2023-07-06T01:20:35.838 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.837+0000 7f61ebfff700 1 -- 172.21.15.22:0/1127192935 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f62040c88c0 con 0x7f62040b40c0 2023-07-06T01:20:35.839 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T01:20:35.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.839+0000 7f6209abe700 1 -- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f61f006ace0 msgr2=0x7f61f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.839+0000 7f6209abe700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f61f006ace0 0x7f61f006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f61f4007fd0 tx=0x7f61f4009040 comp rx=0 tx=0).stop 2023-07-06T01:20:35.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.839+0000 7f6209abe700 1 -- 172.21.15.22:0/1127192935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040b40c0 msgr2=0x7f62040b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:35.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.839+0000 7f6209abe700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040b40c0 0x7f62040b6dd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00cc70 tx=0x7f61fc00cf80 comp rx=0 tx=0).stop 2023-07-06T01:20:35.841 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.839+0000 7f6209abe700 1 -- 172.21.15.22:0/1127192935 shutdown_connections 2023-07-06T01:20:35.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.839+0000 7f6209abe700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040b40c0 0x7f62040b6dd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.839+0000 7f6209abe700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f62040b36f0 0x7f62040ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.839+0000 7f6209abe700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f61f006ace0 0x7f61f006d190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.839+0000 7f6209abe700 1 --2- 172.21.15.22:0/1127192935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62040b3b70 0x7f62040cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:35.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.839+0000 7f6209abe700 1 -- 172.21.15.22:0/1127192935 >> 172.21.15.22:0/1127192935 conn(0x7f620409c2e0 msgr2=0x7f62040d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:35.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.839+0000 7f6209abe700 1 -- 172.21.15.22:0/1127192935 shutdown_connections 2023-07-06T01:20:35.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:35.839+0000 7f6209abe700 1 -- 172.21.15.22:0/1127192935 wait complete. 2023-07-06T01:20:35.853 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-06T01:20:35.853 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T01:20:35.854 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2023-07-06T01:20:35.854 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T01:20:35.854 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T01:20:35.854 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-06T01:20:35.855 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-07-06T01:20:35.855 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T01:20:35.855 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T01:20:35.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:35.856 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2023-07-06T01:20:35.874 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T01:20:35.874 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T01:20:35.874 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T01:20:35.874 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:20:35.875 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2023-07-06T01:20:35.875 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:20:35.875 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:20:35.875 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2023-07-06T01:20:35.876 INFO:teuthology.orchestra.run.smithi022.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T01:20:35.876 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2023-07-06T01:20:35.876 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-06T01:20:35.877 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42225sec preferred_lft 42225sec 2023-07-06T01:20:35.877 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2023-07-06T01:20:35.877 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:20:35.877 INFO:teuthology.orchestra.run.smithi022.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:20:35.877 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2023-07-06T01:20:35.878 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T01:20:35.878 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2023-07-06T01:20:35.878 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T01:20:35.878 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 32:fb:2f:9c:8b:1f brd ff:ff:ff:ff:ff:ff 2023-07-06T01:20:35.879 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T01:20:35.879 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:20:35.879 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::b40c:beff:fe3c:1e8e/64 scope link 2023-07-06T01:20:35.879 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:20:35.879 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T01:20:35.880 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 32:fb:2f:9c:8b:1f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T01:20:35.880 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::30fb:2fff:fe9c:8b1f/64 scope link 2023-07-06T01:20:35.880 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:20:35.880 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T01:20:35.880 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 7a:98:21:a1:32:80 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T01:20:35.881 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::7898:21ff:fea1:3280/64 scope link 2023-07-06T01:20:35.881 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2023-07-06T01:20:35.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:35.882 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T01:20:35.940 INFO:teuthology.orchestra.run.smithi022.stdout:1 2023-07-06T01:20:35.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:35.942 DEBUG:teuthology.orchestra.run.smithi022:> route 2023-07-06T01:20:36.000 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2023-07-06T01:20:36.000 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T01:20:36.000 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-06T01:20:36.000 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-06T01:20:36.000 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T01:20:36.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:36.001 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec bash -c ' 2023-07-06T01:20:36.002 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2023-07-06T01:20:36.002 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-06T01:20:36.002 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-06T01:20:36.003 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-06T01:20:36.003 DEBUG:teuthology.orchestra.run.smithi022:> ') 2023-07-06T01:20:36.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:36.107 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-07-06T01:20:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T01:20:36.124 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T01:20:36.125 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T01:20:36.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:36.125 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2023-07-06T01:20:36.192 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T01:20:36.193 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T01:20:36.250 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T01:20:36.251 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T01:20:36.251 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T01:20:36.307 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-07-06T01:20:36.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:36.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:20:36.393 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:20:36.393 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:20:36.424 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:20:36.425 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:20:36.425 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.103:3300,172.21.15.103:3301" 2023-07-06T01:20:36.425 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with new device syntax: 0@63a894dd-9157-4ce0-adf3-c1bdfe16ec64.cephfs=/ 2023-07-06T01:20:36.425 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.22:3300/172.21.15.103:3300/172.21.15.103:3301" will pass to kernel 2023-07-06T01:20:36.425 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: v2 mount support check returned 2 2023-07-06T01:20:36.425 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-06T01:20:36.426 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T01:20:36.499 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-07-06T01:20:36.499 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-07-06T01:20:36.499 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.103:3300,172.21.15.103:3301" 2023-07-06T01:20:36.499 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with new device syntax: 0@63a894dd-9157-4ce0-adf3-c1bdfe16ec64.cephfs=/ 2023-07-06T01:20:36.500 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.22:3300/172.21.15.103:3300/172.21.15.103:3301" will pass to kernel 2023-07-06T01:20:36.500 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: v2 mount support check returned 2 2023-07-06T01:20:36.500 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-06T01:20:36.500 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2023-07-06T01:20:36.500 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.103:6789,172.21.15.103:6790" 2023-07-06T01:20:36.500 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.103:6789,172.21.15.103:6790:/ 2023-07-06T01:20:36.500 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=63a894dd-9157-4ce0-adf3-c1bdfe16ec64" will pass to kernel 2023-07-06T01:20:36.501 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-06T01:20:36.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:36.502 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T01:20:36.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:36.530 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-06T01:20:36.615 INFO:teuthology.orchestra.run.smithi022.stdout:client4904 2023-07-06T01:20:36.616 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:20:36.617 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:20:36.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2023-07-06T01:20:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.802+0000 7f2c6898b700 1 Processor -- start 2023-07-06T01:20:36.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.802+0000 7f2c6898b700 1 -- start start 2023-07-06T01:20:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.803+0000 7f2c6898b700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f2c641597a0 0x7f2c6406cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.803+0000 7f2c6898b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c640cc750 0x7f2c640cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:36.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.803+0000 7f2c6898b700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c640ccf30 con 0x7f2c640c9a50 2023-07-06T01:20:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.803+0000 7f2c6898b700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c640cd210 con 0x7f2c641597a0 2023-07-06T01:20:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.803+0000 7f2c6898b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640cd4f0 con 0x7f2c640cc750 2023-07-06T01:20:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.803+0000 7f2c62d9d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f2c640c9a50 0x7f2c6406fca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38348/0 (socket says 172.21.15.22:38348) 2023-07-06T01:20:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.803+0000 7f2c62d9d700 1 -- 172.21.15.22:0/586151205 learned_addr learned my addr 172.21.15.22:0/586151205 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:20:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.803+0000 7f2c61d9b700 1 --2- 172.21.15.22:0/586151205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c640cc750 0x7f2c640cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6259c700 1 --2- 172.21.15.22:0/586151205 >> v2:172.21.15.103:3300/0 conn(0x7f2c641597a0 0x7f2c6406cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:36.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319643638 0 0) 0x7f2c640ccf30 con 0x7f2c640c9a50 2023-07-06T01:20:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c4c0035b0 con 0x7f2c640c9a50 2023-07-06T01:20:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3722459236 0 0) 0x7f2c4c0035b0 con 0x7f2c640c9a50 2023-07-06T01:20:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c640cc750 msgr2=0x7f2c640cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 --2- 172.21.15.22:0/586151205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c640cc750 0x7f2c640cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 >> v2:172.21.15.103:3300/0 conn(0x7f2c641597a0 msgr2=0x7f2c6406cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:36.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 --2- 172.21.15.22:0/586151205 >> v2:172.21.15.103:3300/0 conn(0x7f2c641597a0 0x7f2c6406cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c640cd7d0 con 0x7f2c640c9a50 2023-07-06T01:20:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f2c5c0019f0 con 0x7f2c640c9a50 2023-07-06T01:20:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 >> v1:172.21.15.22:6789/0 conn(0x7f2c640c9a50 legacy=0x7f2c6406fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 --2- 172.21.15.22:0/586151205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c4c003a00 0x7f2c4c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 --2- 172.21.15.22:0/586151205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c640cc750 0x7f2c4c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 --2- 172.21.15.22:0/586151205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c4c004d90 0x7f2c4c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:36.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640cd210 con 0x7f2c4c003a00 2023-07-06T01:20:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640cd4f0 con 0x7f2c4c004d90 2023-07-06T01:20:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c00ba10 con 0x7f2c640cc750 2023-07-06T01:20:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6259c700 1 --2- 172.21.15.22:0/586151205 >> v2:172.21.15.103:3300/0 conn(0x7f2c641597a0 0x7f2c6406cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:20:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c61d9b700 1 --2- 172.21.15.22:0/586151205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c640cc750 0x7f2c4c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6259c700 1 --2- 172.21.15.22:0/586151205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c4c003a00 0x7f2c4c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c62d9d700 1 --2- 172.21.15.22:0/586151205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c4c004d90 0x7f2c4c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6259c700 1 -- 172.21.15.22:0/586151205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c640cc750 msgr2=0x7f2c4c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:36.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6259c700 1 --2- 172.21.15.22:0/586151205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c640cc750 0x7f2c4c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6259c700 1 -- 172.21.15.22:0/586151205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c4c004d90 msgr2=0x7f2c4c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:20:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6259c700 1 --2- 172.21.15.22:0/586151205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c4c004d90 0x7f2c4c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.804+0000 7f2c6259c700 1 -- 172.21.15.22:0/586151205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c0035b0 con 0x7f2c4c003a00 2023-07-06T01:20:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.805+0000 7f2c6259c700 1 --2- 172.21.15.22:0/586151205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c4c003a00 0x7f2c4c003e10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009d10 tx=0x7f2c4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540a0edb7083fac0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.805+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58009e60 con 0x7f2c4c003a00 2023-07-06T01:20:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.805+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5800e9a0 con 0x7f2c4c003a00 2023-07-06T01:20:36.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.805+0000 7f2c6159a700 1 -- 172.21.15.22:0/586151205 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800ed10 con 0x7f2c4c003a00 2023-07-06T01:20:36.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.805+0000 7f2c6898b700 1 -- 172.21.15.22:0/586151205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c4c003a00 msgr2=0x7f2c4c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:36.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.805+0000 7f2c6898b700 1 --2- 172.21.15.22:0/586151205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c4c003a00 0x7f2c4c003e10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009d10 tx=0x7f2c4c005880 comp rx=0 tx=0).stop 2023-07-06T01:20:36.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.805+0000 7f2c62d9d700 1 -- 172.21.15.22:0/586151205 reap_dead start 2023-07-06T01:20:36.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.805+0000 7f2c6898b700 1 -- 172.21.15.22:0/586151205 shutdown_connections 2023-07-06T01:20:36.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.805+0000 7f2c6898b700 1 -- 172.21.15.22:0/586151205 >> 172.21.15.22:0/586151205 conn(0x7f2c640b0710 msgr2=0x7f2c640b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:36.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.805+0000 7f2c6898b700 1 -- 172.21.15.22:0/586151205 shutdown_connections 2023-07-06T01:20:36.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.805+0000 7f2c6898b700 1 -- 172.21.15.22:0/586151205 wait complete. 2023-07-06T01:20:36.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.807+0000 7f2c6898b700 1 Processor -- start 2023-07-06T01:20:36.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.807+0000 7f2c6898b700 1 -- start start 2023-07-06T01:20:36.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c6898b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c4c003a00 0x7f2c64150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:36.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c6898b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c641509c0 0x7f2c6415ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:36.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c6898b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c640b2b90 0x7f2c640b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:36.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c6898b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c64158610 con 0x7f2c640b2b90 2023-07-06T01:20:36.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c6898b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c64159f20 con 0x7f2c641509c0 2023-07-06T01:20:36.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c6898b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640b3540 con 0x7f2c4c003a00 2023-07-06T01:20:36.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c62d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c640b2b90 0x7f2c640b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:36.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c62d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c640b2b90 0x7f2c640b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47616/0 (socket says 172.21.15.22:47616) 2023-07-06T01:20:36.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c62d9d700 1 -- 172.21.15.22:0/758977950 learned_addr learned my addr 172.21.15.22:0/758977950 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:20:36.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c61d9b700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c641509c0 0x7f2c6415ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:36.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c6259c700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c4c003a00 0x7f2c64150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:36.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c62d9d700 1 -- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c4c003a00 msgr2=0x7f2c64150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:36.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c62d9d700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c4c003a00 0x7f2c64150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:36.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c62d9d700 1 -- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c641509c0 msgr2=0x7f2c6415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:36.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c62d9d700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c641509c0 0x7f2c6415ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:36.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c62d9d700 1 -- 172.21.15.22:0/758977950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64158820 con 0x7f2c640b2b90 2023-07-06T01:20:36.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c62d9d700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c640b2b90 0x7f2c640b3000 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0019e0 tx=0x7f2c5c00a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:36.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c52ffd700 1 -- 172.21.15.22:0/758977950 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00ad00 con 0x7f2c640b2b90 2023-07-06T01:20:36.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c52ffd700 1 -- 172.21.15.22:0/758977950 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c00ae60 con 0x7f2c640b2b90 2023-07-06T01:20:36.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c6898b700 1 -- 172.21.15.22:0/758977950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c64158b00 con 0x7f2c640b2b90 2023-07-06T01:20:36.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.808+0000 7f2c52ffd700 1 -- 172.21.15.22:0/758977950 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c013630 con 0x7f2c640b2b90 2023-07-06T01:20:36.818 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.809+0000 7f2c6898b700 1 -- 172.21.15.22:0/758977950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c641590c0 con 0x7f2c640b2b90 2023-07-06T01:20:36.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.809+0000 7f2c6898b700 1 -- 172.21.15.22:0/758977950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c640744b0 con 0x7f2c640b2b90 2023-07-06T01:20:36.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.810+0000 7f2c52ffd700 1 -- 172.21.15.22:0/758977950 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2c5c013790 con 0x7f2c640b2b90 2023-07-06T01:20:36.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.810+0000 7f2c52ffd700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2c4406ace0 0x7f2c4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:20:36.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.810+0000 7f2c52ffd700 1 -- 172.21.15.22:0/758977950 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2c5c089b40 con 0x7f2c640b2b90 2023-07-06T01:20:36.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.810+0000 7f2c6259c700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2c4406ace0 0x7f2c4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:20:36.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.811+0000 7f2c6259c700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2c4406ace0 0x7f2c4406d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2c580090b0 tx=0x7f2c58015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:20:36.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:36.815+0000 7f2c52ffd700 1 -- 172.21.15.22:0/758977950 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2c5c058680 con 0x7f2c640b2b90 2023-07-06T01:20:37.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.059+0000 7f2c6898b700 1 -- 172.21.15.22:0/758977950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2c640c9040 con 0x7f2c640b2b90 2023-07-06T01:20:37.182 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.180+0000 7f2c52ffd700 1 -- 172.21.15.22:0/758977950 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v16) v1 ==== 167+0+0 (secure 0 0 0) 0x7f2c5c058210 con 0x7f2c640b2b90 2023-07-06T01:20:37.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.183+0000 7f2c6898b700 1 -- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2c4406ace0 msgr2=0x7f2c4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:37.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.183+0000 7f2c6898b700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2c4406ace0 0x7f2c4406d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2c580090b0 tx=0x7f2c58015000 comp rx=0 tx=0).stop 2023-07-06T01:20:37.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.183+0000 7f2c6898b700 1 -- 172.21.15.22:0/758977950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c640b2b90 msgr2=0x7f2c640b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:20:37.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.183+0000 7f2c6898b700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c640b2b90 0x7f2c640b3000 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0019e0 tx=0x7f2c5c00a640 comp rx=0 tx=0).stop 2023-07-06T01:20:37.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.183+0000 7f2c6898b700 1 -- 172.21.15.22:0/758977950 shutdown_connections 2023-07-06T01:20:37.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.183+0000 7f2c6898b700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c640b2b90 0x7f2c640b3000 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:37.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.183+0000 7f2c6898b700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c4c003a00 0x7f2c64150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:37.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.183+0000 7f2c6898b700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2c4406ace0 0x7f2c4406d190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:37.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.183+0000 7f2c6898b700 1 --2- 172.21.15.22:0/758977950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c641509c0 0x7f2c6415ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:20:37.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.183+0000 7f2c6898b700 1 -- 172.21.15.22:0/758977950 >> 172.21.15.22:0/758977950 conn(0x7f2c640b0710 msgr2=0x7f2c640b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:20:37.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.183+0000 7f2c6898b700 1 -- 172.21.15.22:0/758977950 shutdown_connections 2023-07-06T01:20:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:20:37.183+0000 7f2c6898b700 1 -- 172.21.15.22:0/758977950 wait complete. 2023-07-06T01:20:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:enabled new snapshots 2023-07-06T01:20:37.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:37.198 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2023-07-06T01:20:37.221 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:20:37.221 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:20:37.280 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T01:20:37.280 DEBUG:teuthology.orchestra.run.smithi022:> import os 2023-07-06T01:20:37.281 DEBUG:teuthology.orchestra.run.smithi022:> import uuid 2023-07-06T01:20:37.281 DEBUG:teuthology.orchestra.run.smithi022:> 2023-07-06T01:20:37.282 DEBUG:teuthology.orchestra.run.smithi022:> n = 2000 2023-07-06T01:20:37.283 DEBUG:teuthology.orchestra.run.smithi022:> create_hard_links = False 2023-07-06T01:20:37.283 DEBUG:teuthology.orchestra.run.smithi022:> if 3 > 0: 2023-07-06T01:20:37.284 DEBUG:teuthology.orchestra.run.smithi022:> create_hard_links = True 2023-07-06T01:20:37.284 DEBUG:teuthology.orchestra.run.smithi022:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2023-07-06T01:20:37.285 DEBUG:teuthology.orchestra.run.smithi022:> 2023-07-06T01:20:37.285 DEBUG:teuthology.orchestra.run.smithi022:> dpath = os.path.dirname(path) 2023-07-06T01:20:37.286 DEBUG:teuthology.orchestra.run.smithi022:> fnameprefix = os.path.basename(path) 2023-07-06T01:20:37.286 DEBUG:teuthology.orchestra.run.smithi022:> os.makedirs(dpath, exist_ok=True) 2023-07-06T01:20:37.287 DEBUG:teuthology.orchestra.run.smithi022:> 2023-07-06T01:20:37.288 DEBUG:teuthology.orchestra.run.smithi022:> try: 2023-07-06T01:20:37.288 DEBUG:teuthology.orchestra.run.smithi022:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-07-06T01:20:37.289 DEBUG:teuthology.orchestra.run.smithi022:> 2023-07-06T01:20:37.289 DEBUG:teuthology.orchestra.run.smithi022:> for i in range(n): 2023-07-06T01:20:37.290 DEBUG:teuthology.orchestra.run.smithi022:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-07-06T01:20:37.290 DEBUG:teuthology.orchestra.run.smithi022:> with open(fpath, '"'"'w'"'"') as f: 2023-07-06T01:20:37.291 DEBUG:teuthology.orchestra.run.smithi022:> f.write(f"{i}") 2023-07-06T01:20:37.291 DEBUG:teuthology.orchestra.run.smithi022:> if False: 2023-07-06T01:20:37.292 DEBUG:teuthology.orchestra.run.smithi022:> f.flush() 2023-07-06T01:20:37.293 DEBUG:teuthology.orchestra.run.smithi022:> os.fsync(f.fileno()) 2023-07-06T01:20:37.293 DEBUG:teuthology.orchestra.run.smithi022:> if False: 2023-07-06T01:20:37.294 DEBUG:teuthology.orchestra.run.smithi022:> os.unlink(fpath) 2023-07-06T01:20:37.294 DEBUG:teuthology.orchestra.run.smithi022:> if False: 2023-07-06T01:20:37.295 DEBUG:teuthology.orchestra.run.smithi022:> os.fsync(dirfd) 2023-07-06T01:20:37.295 DEBUG:teuthology.orchestra.run.smithi022:> if create_hard_links: 2023-07-06T01:20:37.296 DEBUG:teuthology.orchestra.run.smithi022:> for j in range(3): 2023-07-06T01:20:37.296 DEBUG:teuthology.orchestra.run.smithi022:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-07-06T01:20:37.297 DEBUG:teuthology.orchestra.run.smithi022:> if False: 2023-07-06T01:20:37.298 DEBUG:teuthology.orchestra.run.smithi022:> os.fsync(dirfd) 2023-07-06T01:20:37.298 DEBUG:teuthology.orchestra.run.smithi022:> finally: 2023-07-06T01:20:37.299 DEBUG:teuthology.orchestra.run.smithi022:> os.close(dirfd) 2023-07-06T01:20:37.299 DEBUG:teuthology.orchestra.run.smithi022:> ' 2023-07-06T01:20:59.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:59.202 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2023-07-06T01:20:59.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:20:59.658 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2023-07-06T01:21:02.027 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:21:02.030 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:21:32.368 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:21:32.370 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:21:35.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:35.301 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2023-07-06T01:21:35.455 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2023-07-06T01:21:35.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.581+0000 7f463c89f700 1 Processor -- start 2023-07-06T01:21:35.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.581+0000 7f463c89f700 1 -- start start 2023-07-06T01:21:35.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.582+0000 7f463c89f700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f46380b36f0 0x7f46380cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.582+0000 7f463c89f700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380cf5a0 0x7f46380d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.582+0000 7f463c89f700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46380cebd0 con 0x7f46380b3b70 2023-07-06T01:21:35.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.582+0000 7f463c89f700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463815cff0 con 0x7f46380b36f0 2023-07-06T01:21:35.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.582+0000 7f463c89f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46380cdfc0 con 0x7f46380cf5a0 2023-07-06T01:21:35.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.582+0000 7f4636d9d700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f46380b3b70 0x7f46380b5da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38384/0 (socket says 172.21.15.22:38384) 2023-07-06T01:21:35.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.582+0000 7f4636d9d700 1 -- 172.21.15.22:0/4289728756 learned_addr learned my addr 172.21.15.22:0/4289728756 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:35.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.582+0000 7f4635d9b700 1 --2- 172.21.15.22:0/4289728756 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380cf5a0 0x7f46380d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.582+0000 7f463659c700 1 --2- 172.21.15.22:0/4289728756 >> v2:172.21.15.103:3300/0 conn(0x7f46380b36f0 0x7f46380cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.582+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156452130 0 0) 0x7f46380cebd0 con 0x7f46380b3b70 2023-07-06T01:21:35.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.582+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46200035b0 con 0x7f46380b3b70 2023-07-06T01:21:35.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3392547977 0 0) 0x7f46200035b0 con 0x7f46380b3b70 2023-07-06T01:21:35.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380cf5a0 msgr2=0x7f46380d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 --2- 172.21.15.22:0/4289728756 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380cf5a0 0x7f46380d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.588 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 >> v2:172.21.15.103:3300/0 conn(0x7f46380b36f0 msgr2=0x7f46380cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 --2- 172.21.15.22:0/4289728756 >> v2:172.21.15.103:3300/0 conn(0x7f46380b36f0 0x7f46380cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380ce2a0 con 0x7f46380b3b70 2023-07-06T01:21:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f46300019f0 con 0x7f46380b3b70 2023-07-06T01:21:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 >> v1:172.21.15.22:6789/0 conn(0x7f46380b3b70 legacy=0x7f46380b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 --2- 172.21.15.22:0/4289728756 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380cf5a0 0x7f4620003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 --2- 172.21.15.22:0/4289728756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4620003f80 0x7f4620004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.589 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 --2- 172.21.15.22:0/4289728756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4620004d90 0x7f4620005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463815cff0 con 0x7f4620004d90 2023-07-06T01:21:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380cdfc0 con 0x7f4620003f80 2023-07-06T01:21:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462000ba40 con 0x7f46380cf5a0 2023-07-06T01:21:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463659c700 1 --2- 172.21.15.22:0/4289728756 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380cf5a0 0x7f4620003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f4636d9d700 1 --2- 172.21.15.22:0/4289728756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4620004d90 0x7f4620005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f4635d9b700 1 --2- 172.21.15.22:0/4289728756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4620003f80 0x7f4620004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463659c700 1 -- 172.21.15.22:0/4289728756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4620003f80 msgr2=0x7f4620004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.590 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463659c700 1 --2- 172.21.15.22:0/4289728756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4620003f80 0x7f4620004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463659c700 1 -- 172.21.15.22:0/4289728756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4620004d90 msgr2=0x7f4620005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463659c700 1 --2- 172.21.15.22:0/4289728756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4620004d90 0x7f4620005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463659c700 1 -- 172.21.15.22:0/4289728756 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46200035b0 con 0x7f46380cf5a0 2023-07-06T01:21:35.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f4636d9d700 1 --2- 172.21.15.22:0/4289728756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4620004d90 0x7f4620005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:35.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.583+0000 7f463659c700 1 --2- 172.21.15.22:0/4289728756 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380cf5a0 0x7f4620003a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f462c009b60 tx=0x7f4620005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80e8fe3a7db147d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.584+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00e660 con 0x7f46380cf5a0 2023-07-06T01:21:35.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.584+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c00e7c0 con 0x7f46380cf5a0 2023-07-06T01:21:35.591 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.584+0000 7f463559a700 1 -- 172.21.15.22:0/4289728756 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00eb30 con 0x7f46380cf5a0 2023-07-06T01:21:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.584+0000 7f463c89f700 1 -- 172.21.15.22:0/4289728756 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380cf5a0 msgr2=0x7f4620003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.584+0000 7f463c89f700 1 --2- 172.21.15.22:0/4289728756 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380cf5a0 0x7f4620003a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f462c009b60 tx=0x7f4620005880 comp rx=0 tx=0).stop 2023-07-06T01:21:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.584+0000 7f4636d9d700 1 -- 172.21.15.22:0/4289728756 reap_dead start 2023-07-06T01:21:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.584+0000 7f463c89f700 1 -- 172.21.15.22:0/4289728756 shutdown_connections 2023-07-06T01:21:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.584+0000 7f463c89f700 1 -- 172.21.15.22:0/4289728756 >> 172.21.15.22:0/4289728756 conn(0x7f463809c2e0 msgr2=0x7f46380d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.584+0000 7f463c89f700 1 -- 172.21.15.22:0/4289728756 shutdown_connections 2023-07-06T01:21:35.593 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.584+0000 7f463c89f700 1 -- 172.21.15.22:0/4289728756 wait complete. 2023-07-06T01:21:35.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.586+0000 7f463c89f700 1 Processor -- start 2023-07-06T01:21:35.596 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.586+0000 7f463c89f700 1 -- start start 2023-07-06T01:21:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.586+0000 7f463c89f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46380cf5a0 0x7f463815fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.586+0000 7f463c89f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46380c75f0 0x7f46381f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.586+0000 7f463c89f700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380c8030 0x7f46380d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.586+0000 7f463c89f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380c7a60 con 0x7f46380c75f0 2023-07-06T01:21:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.586+0000 7f463c89f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46380c87d0 con 0x7f46380cf5a0 2023-07-06T01:21:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.586+0000 7f463c89f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46380ca0c0 con 0x7f46380c8030 2023-07-06T01:21:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f4635d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46380c75f0 0x7f46381f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f4635d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46380c75f0 0x7f46381f3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47652/0 (socket says 172.21.15.22:47652) 2023-07-06T01:21:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f4635d9b700 1 -- 172.21.15.22:0/3306830851 learned_addr learned my addr 172.21.15.22:0/3306830851 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f463659c700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46380cf5a0 0x7f463815fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f4636d9d700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380c8030 0x7f46380d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f4635d9b700 1 -- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380c8030 msgr2=0x7f46380d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f4635d9b700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380c8030 0x7f46380d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f4635d9b700 1 -- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46380cf5a0 msgr2=0x7f463815fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f4635d9b700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46380cf5a0 0x7f463815fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f4635d9b700 1 -- 172.21.15.22:0/3306830851 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380ca3a0 con 0x7f46380c75f0 2023-07-06T01:21:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f4635d9b700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46380c75f0 0x7f46381f3520 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4628011aa0 tx=0x7f4628011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f4626ffd700 1 -- 172.21.15.22:0/3306830851 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462800f8e0 con 0x7f46380c75f0 2023-07-06T01:21:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f4626ffd700 1 -- 172.21.15.22:0/3306830851 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462800fa40 con 0x7f46380c75f0 2023-07-06T01:21:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f463c89f700 1 -- 172.21.15.22:0/3306830851 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46380ca680 con 0x7f46380c75f0 2023-07-06T01:21:35.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f4626ffd700 1 -- 172.21.15.22:0/3306830851 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462800fdf0 con 0x7f46380c75f0 2023-07-06T01:21:35.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.587+0000 7f463c89f700 1 -- 172.21.15.22:0/3306830851 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46380c84a0 con 0x7f46380c75f0 2023-07-06T01:21:35.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.589+0000 7f4626ffd700 1 -- 172.21.15.22:0/3306830851 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4628014440 con 0x7f46380c75f0 2023-07-06T01:21:35.600 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.589+0000 7f4626ffd700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f461806ace0 0x7f461806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.589+0000 7f4626ffd700 1 -- 172.21.15.22:0/3306830851 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f4628089a70 con 0x7f46380c75f0 2023-07-06T01:21:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.589+0000 7f463659c700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f461806ace0 0x7f461806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.590+0000 7f463659c700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f461806ace0 0x7f461806d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f462c0090b0 tx=0x7f462c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.591+0000 7f4648a24b80 1 Processor -- start 2023-07-06T01:21:35.601 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.592+0000 7f4648a24b80 1 -- start start 2023-07-06T01:21:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.592+0000 7f4648a24b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 0x562b340886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.592+0000 7f4648a24b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b3403f870 0x562b3406c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.592+0000 7f4648a24b80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b34069d10 0x562b3406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.592+0000 7f4648a24b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b33f46ca0 con 0x562b340694b0 2023-07-06T01:21:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.592+0000 7f4648a24b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b33f99a30 con 0x562b3403f870 2023-07-06T01:21:35.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.592+0000 7f4648a24b80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b33f92ec0 con 0x562b34069d10 2023-07-06T01:21:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.592+0000 7f463659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 0x562b340886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.592+0000 7f463659c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 0x562b340886b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47656/0 (socket says 172.21.15.22:47656) 2023-07-06T01:21:35.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.592+0000 7f463659c700 1 -- 172.21.15.22:0/677831168 learned_addr learned my addr 172.21.15.22:0/677831168 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:35.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.593+0000 7f4636d9d700 1 --2- 172.21.15.22:0/677831168 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b34069d10 0x562b3406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.593+0000 7f4635d9b700 1 --2- 172.21.15.22:0/677831168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b3403f870 0x562b3406c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.593+0000 7f463659c700 1 -- 172.21.15.22:0/677831168 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b34069d10 msgr2=0x562b3406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.605 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.593+0000 7f463659c700 1 --2- 172.21.15.22:0/677831168 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b34069d10 0x562b3406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.593+0000 7f463659c700 1 -- 172.21.15.22:0/677831168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b3403f870 msgr2=0x562b3406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.593+0000 7f463659c700 1 --2- 172.21.15.22:0/677831168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b3403f870 0x562b3406c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.593+0000 7f463659c700 1 -- 172.21.15.22:0/677831168 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b33dc0dd0 con 0x562b340694b0 2023-07-06T01:21:35.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.593+0000 7f463659c700 1 --2- 172.21.15.22:0/677831168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 0x562b340886b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f462c00ea30 tx=0x7f462c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3090c5f61f63d89 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.593+0000 7f4617fff700 1 -- 172.21.15.22:0/677831168 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c004230 con 0x562b340694b0 2023-07-06T01:21:35.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.593+0000 7f4617fff700 1 -- 172.21.15.22:0/677831168 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c022040 con 0x562b340694b0 2023-07-06T01:21:35.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.593+0000 7f4617fff700 1 -- 172.21.15.22:0/677831168 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c0045d0 con 0x562b340694b0 2023-07-06T01:21:35.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.594+0000 7f4648a24b80 1 -- 172.21.15.22:0/677831168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 msgr2=0x562b340886b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.594+0000 7f4648a24b80 1 --2- 172.21.15.22:0/677831168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 0x562b340886b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f462c00ea30 tx=0x7f462c013000 comp rx=0 tx=0).stop 2023-07-06T01:21:35.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.594+0000 7f4648a24b80 1 -- 172.21.15.22:0/677831168 shutdown_connections 2023-07-06T01:21:35.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.594+0000 7f4648a24b80 1 --2- 172.21.15.22:0/677831168 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b34069d10 0x562b3406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.594+0000 7f4648a24b80 1 --2- 172.21.15.22:0/677831168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 0x562b340886b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.594+0000 7f4648a24b80 1 --2- 172.21.15.22:0/677831168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b3403f870 0x562b3406c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.594+0000 7f4648a24b80 1 -- 172.21.15.22:0/677831168 >> 172.21.15.22:0/677831168 conn(0x562b34036ec0 msgr2=0x562b34087230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:35.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.594+0000 7f4648a24b80 1 -- 172.21.15.22:0/677831168 shutdown_connections 2023-07-06T01:21:35.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.594+0000 7f4648a24b80 1 -- 172.21.15.22:0/677831168 wait complete. 2023-07-06T01:21:35.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.594+0000 7f4648a24b80 1 -- start start 2023-07-06T01:21:35.610 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.594+0000 7f4648a24b80 1 Processor -- start 2023-07-06T01:21:35.614 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.595+0000 7f4648a24b80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b3403f870 0x562b34064af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.595+0000 7f4648a24b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 0x562b3406a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.595+0000 7f4648a24b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b34069d10 0x562b3408a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.595+0000 7f4648a24b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b33f32ab0 con 0x562b340694b0 2023-07-06T01:21:35.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.595+0000 7f4648a24b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b33f327b0 con 0x562b34069d10 2023-07-06T01:21:35.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.595+0000 7f4648a24b80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b33e16910 con 0x562b3403f870 2023-07-06T01:21:35.615 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.595+0000 7f4648a24b80 10 client.0 _ll_register_callbacks cb 0x562b33f30750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T01:21:35.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.595+0000 7f4635d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 0x562b3406a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f463659c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b3403f870 0x562b34064af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f4635d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 0x562b3406a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47664/0 (socket says 172.21.15.22:47664) 2023-07-06T01:21:35.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f4635d9b700 1 -- 172.21.15.22:0/1099514178 learned_addr learned my addr 172.21.15.22:0/1099514178 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:35.616 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f4636d9d700 1 --2- 172.21.15.22:0/1099514178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b34069d10 0x562b3408a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f4635d9b700 1 -- 172.21.15.22:0/1099514178 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b3403f870 msgr2=0x562b34064af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f4635d9b700 1 --2- 172.21.15.22:0/1099514178 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b3403f870 0x562b34064af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f4635d9b700 1 -- 172.21.15.22:0/1099514178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b34069d10 msgr2=0x562b3408a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f4635d9b700 1 --2- 172.21.15.22:0/1099514178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b34069d10 0x562b3408a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f4635d9b700 1 -- 172.21.15.22:0/1099514178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46200035b0 con 0x562b340694b0 2023-07-06T01:21:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f463659c700 1 --2- 172.21.15.22:0/1099514178 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b3403f870 0x562b34064af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:35.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f4635d9b700 1 --2- 172.21.15.22:0/1099514178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 0x562b3406a180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4628087eb0 tx=0x7f462805c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09c3b80ff9fff3b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f46157fa700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-07-06T01:21:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f46157fa700 1 -- 172.21.15.22:0/1099514178 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4628060910 con 0x562b340694b0 2023-07-06T01:21:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f46157fa700 1 -- 172.21.15.22:0/1099514178 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f4628086020 con 0x562b340694b0 2023-07-06T01:21:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f4648a24b80 1 -- 172.21.15.22:0/1099514178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b340344d0 con 0x562b340694b0 2023-07-06T01:21:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f46157fa700 1 -- 172.21.15.22:0/1099514178 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4628054640 con 0x562b340694b0 2023-07-06T01:21:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f4648a24b80 10 client.5343 fetch_fsmap learned FSMap version 23 2023-07-06T01:21:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f4648a24b80 1 -- 172.21.15.22:0/1099514178 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x562b33da9c50 con 0x562b340694b0 2023-07-06T01:21:35.618 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f46157fa700 1 -- 172.21.15.22:0/1099514178 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b340344d0 con 0x562b340694b0 2023-07-06T01:21:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.596+0000 7f46157fa700 1 -- 172.21.15.22:0/1099514178 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f46280547a0 con 0x562b340694b0 2023-07-06T01:21:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.597+0000 7f4648a24b80 10 client.5343 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T01:21:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.597+0000 7f4648a24b80 10 client.5343 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-07-06T01:21:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.597+0000 7f4648a24b80 20 client.5343 populate_metadata read hostname 'smithi022' 2023-07-06T01:21:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.597+0000 7f4648a24b80 1 --2- 172.21.15.22:0/1099514178 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x562b3406a6c0 0x562b34081810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.597+0000 7f4648a24b80 4 client.5343 mds_command: new command op to 5115 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T01:21:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.597+0000 7f4648a24b80 1 -- 172.21.15.22:0/1099514178 --> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b33f8bea0 con 0x562b3406a6c0 2023-07-06T01:21:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.597+0000 7f463659c700 1 --2- 172.21.15.22:0/1099514178 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x562b3406a6c0 0x562b34081810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.597+0000 7f46157fa700 10 client.5343 ms_handle_connect on v2:172.21.15.22:6833/612827809 2023-07-06T01:21:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.597+0000 7f463659c700 1 --2- 172.21.15.22:0/1099514178 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x562b3406a6c0 0x562b34081810 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f34af6f7b022cfc server_cookie=681fe024e8eb6cbf in_seq=0 out_seq=0 2023-07-06T01:21:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.597+0000 7f46157fa700 1 -- 172.21.15.22:0/1099514178 <== mds.0 v2:172.21.15.22:6833/612827809 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562b33f8bea0 con 0x562b3406a6c0 2023-07-06T01:21:35.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.597+0000 7f46157fa700 10 client.5343 handle_command_reply: tid=0 2023-07-06T01:21:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.598+0000 7f4648a24b80 1 client.5343 shutdown 2023-07-06T01:21:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.598+0000 7f4648a24b80 1 -- 172.21.15.22:0/1099514178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 msgr2=0x562b3406a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.598+0000 7f4648a24b80 1 --2- 172.21.15.22:0/1099514178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 0x562b3406a180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4628087eb0 tx=0x7f462805c810 comp rx=0 tx=0).stop 2023-07-06T01:21:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.598+0000 7f4648a24b80 1 -- 172.21.15.22:0/1099514178 shutdown_connections 2023-07-06T01:21:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.598+0000 7f4648a24b80 1 --2- 172.21.15.22:0/1099514178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b34069d10 0x562b3408a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.598+0000 7f4648a24b80 1 --2- 172.21.15.22:0/1099514178 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b3403f870 0x562b34064af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.598+0000 7f4648a24b80 1 --2- 172.21.15.22:0/1099514178 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x562b3406a6c0 0x562b34081810 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.598+0000 7f4648a24b80 1 --2- 172.21.15.22:0/1099514178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b340694b0 0x562b3406a180 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.598+0000 7f46157fa700 0 client.5343 ms_handle_reset on v2:172.21.15.22:6833/612827809 2023-07-06T01:21:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.598+0000 7f4648a24b80 1 -- 172.21.15.22:0/1099514178 >> 172.21.15.22:0/1099514178 conn(0x562b34036ec0 msgr2=0x562b340673a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.598+0000 7f4648a24b80 1 -- 172.21.15.22:0/1099514178 shutdown_connections 2023-07-06T01:21:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.598+0000 7f4648a24b80 1 -- 172.21.15.22:0/1099514178 wait complete. 2023-07-06T01:21:35.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.598+0000 7f4648a24b80 20 client.5343 trim_cache size 0 max 16384 2023-07-06T01:21:35.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.614+0000 7f4648a24b80 1 Processor -- start 2023-07-06T01:21:35.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.614+0000 7f4648a24b80 1 -- start start 2023-07-06T01:21:35.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4648a24b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b3403f870 0x562b34067a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4648a24b80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b340694b0 0x562b3403e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4648a24b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 0x562b34068220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4648a24b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b340643b0 con 0x562b34069d10 2023-07-06T01:21:35.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4648a24b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34036a20 con 0x562b3403f870 2023-07-06T01:21:35.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4648a24b80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b3406a1c0 con 0x562b340694b0 2023-07-06T01:21:35.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4636d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 0x562b34068220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4636d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 0x562b34068220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47674/0 (socket says 172.21.15.22:47674) 2023-07-06T01:21:35.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4636d9d700 1 -- 172.21.15.22:0/2035106912 learned_addr learned my addr 172.21.15.22:0/2035106912 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:35.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f463659c700 1 --2- 172.21.15.22:0/2035106912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b3403f870 0x562b34067a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4635d9b700 1 --2- 172.21.15.22:0/2035106912 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b340694b0 0x562b3403e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4636d9d700 1 -- 172.21.15.22:0/2035106912 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b340694b0 msgr2=0x562b3403e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4636d9d700 1 --2- 172.21.15.22:0/2035106912 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b340694b0 0x562b3403e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4636d9d700 1 -- 172.21.15.22:0/2035106912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b3403f870 msgr2=0x562b34067a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4636d9d700 1 --2- 172.21.15.22:0/2035106912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b3403f870 0x562b34067a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4636d9d700 1 -- 172.21.15.22:0/2035106912 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b33de7240 con 0x562b34069d10 2023-07-06T01:21:35.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4636d9d700 1 --2- 172.21.15.22:0/2035106912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 0x562b34068220 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4630006010 tx=0x7f4630006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fd0c7db68c7ee5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4624ff9700 1 -- 172.21.15.22:0/2035106912 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630018650 con 0x562b34069d10 2023-07-06T01:21:35.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4624ff9700 1 -- 172.21.15.22:0/2035106912 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463001c070 con 0x562b34069d10 2023-07-06T01:21:35.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.615+0000 7f4624ff9700 1 -- 172.21.15.22:0/2035106912 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630018a10 con 0x562b34069d10 2023-07-06T01:21:35.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.616+0000 7f4648a24b80 1 -- 172.21.15.22:0/2035106912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 msgr2=0x562b34068220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.616+0000 7f4648a24b80 1 --2- 172.21.15.22:0/2035106912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 0x562b34068220 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4630006010 tx=0x7f4630006040 comp rx=0 tx=0).stop 2023-07-06T01:21:35.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.616+0000 7f4648a24b80 1 -- 172.21.15.22:0/2035106912 shutdown_connections 2023-07-06T01:21:35.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.616+0000 7f4648a24b80 1 --2- 172.21.15.22:0/2035106912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 0x562b34068220 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.616+0000 7f4648a24b80 1 --2- 172.21.15.22:0/2035106912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b3403f870 0x562b34067a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.616+0000 7f4648a24b80 1 --2- 172.21.15.22:0/2035106912 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b340694b0 0x562b3403e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.616+0000 7f4648a24b80 1 -- 172.21.15.22:0/2035106912 >> 172.21.15.22:0/2035106912 conn(0x562b34036ec0 msgr2=0x562b34083d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:35.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.616+0000 7f4648a24b80 1 -- 172.21.15.22:0/2035106912 shutdown_connections 2023-07-06T01:21:35.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.616+0000 7f4648a24b80 1 -- 172.21.15.22:0/2035106912 wait complete. 2023-07-06T01:21:35.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.616+0000 7f4648a24b80 1 -- start start 2023-07-06T01:21:35.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.617+0000 7f4648a24b80 1 Processor -- start 2023-07-06T01:21:35.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4648a24b80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b3403f870 0x562b34068220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4648a24b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b340694b0 0x562b34068760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4648a24b80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 0x562b34082b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4648a24b80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b34087ff0 con 0x562b34069d10 2023-07-06T01:21:35.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4648a24b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b340a0530 con 0x562b340694b0 2023-07-06T01:21:35.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4648a24b80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b3406dc70 con 0x562b3403f870 2023-07-06T01:21:35.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4648a24b80 10 client.0 _ll_register_callbacks cb 0x7f46280586b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T01:21:35.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4636d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 0x562b34082b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4636d9d700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 0x562b34082b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47680/0 (socket says 172.21.15.22:47680) 2023-07-06T01:21:35.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4636d9d700 1 -- 172.21.15.22:0/1452307042 learned_addr learned my addr 172.21.15.22:0/1452307042 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:35.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f463659c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b3403f870 0x562b34068220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4635d9b700 1 --2- 172.21.15.22:0/1452307042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b340694b0 0x562b34068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4636d9d700 1 -- 172.21.15.22:0/1452307042 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b3403f870 msgr2=0x562b34068220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.633 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4636d9d700 1 --2- 172.21.15.22:0/1452307042 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b3403f870 0x562b34068220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4636d9d700 1 -- 172.21.15.22:0/1452307042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b340694b0 msgr2=0x562b34068760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4636d9d700 1 --2- 172.21.15.22:0/1452307042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b340694b0 0x562b34068760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.618+0000 7f4636d9d700 1 -- 172.21.15.22:0/1452307042 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46380ce2a0 con 0x562b34069d10 2023-07-06T01:21:35.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f4636d9d700 1 --2- 172.21.15.22:0/1452307042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 0x562b34082b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f463808bbb0 tx=0x7f4630001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9363b0de9e58726 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f46157fa700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-07-06T01:21:35.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f46157fa700 1 -- 172.21.15.22:0/1452307042 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463000a340 con 0x562b34069d10 2023-07-06T01:21:35.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f4648a24b80 1 -- 172.21.15.22:0/1452307042 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f462c022040 con 0x562b34069d10 2023-07-06T01:21:35.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f46157fa700 1 -- 172.21.15.22:0/1452307042 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f4630022430 con 0x562b34069d10 2023-07-06T01:21:35.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f46157fa700 1 -- 172.21.15.22:0/1452307042 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4630018e70 con 0x562b34069d10 2023-07-06T01:21:35.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f4648a24b80 10 client.5349 fetch_fsmap learned FSMap version 23 2023-07-06T01:21:35.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f4648a24b80 1 -- 172.21.15.22:0/1452307042 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x562b33dbf2a0 con 0x562b34069d10 2023-07-06T01:21:35.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f46157fa700 1 -- 172.21.15.22:0/1452307042 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c022040 con 0x562b34069d10 2023-07-06T01:21:35.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f46157fa700 1 -- 172.21.15.22:0/1452307042 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4630012070 con 0x562b34069d10 2023-07-06T01:21:35.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f4648a24b80 10 client.5349 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T01:21:35.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f4648a24b80 10 client.5349 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-07-06T01:21:35.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f4648a24b80 20 client.5349 populate_metadata read hostname 'smithi022' 2023-07-06T01:21:35.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f4648a24b80 1 --2- 172.21.15.22:0/1452307042 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x562b340d0ae0 0x562b340830d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f4648a24b80 4 client.5349 mds_command: new command op to 5115 tid=0[{"prefix": "perf dump"}] 2023-07-06T01:21:35.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f4648a24b80 1 -- 172.21.15.22:0/1452307042 --> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x562b33f4e040 con 0x562b340d0ae0 2023-07-06T01:21:35.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.619+0000 7f463659c700 1 --2- 172.21.15.22:0/1452307042 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x562b340d0ae0 0x562b340830d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f46157fa700 10 client.5349 ms_handle_connect on v2:172.21.15.22:6833/612827809 2023-07-06T01:21:35.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f463659c700 1 --2- 172.21.15.22:0/1452307042 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x562b340d0ae0 0x562b340830d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8168883d65f0e8ea server_cookie=88895dc342b23af6 in_seq=0 out_seq=0 2023-07-06T01:21:35.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f46157fa700 1 -- 172.21.15.22:0/1452307042 <== mds.0 v2:172.21.15.22:6833/612827809 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17851 (crc 0 0 0) 0x562b33f4e040 con 0x562b340d0ae0 2023-07-06T01:21:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f46157fa700 10 client.5349 handle_command_reply: tid=0 2023-07-06T01:21:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f4648a24b80 1 client.5349 shutdown 2023-07-06T01:21:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f4648a24b80 1 -- 172.21.15.22:0/1452307042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 msgr2=0x562b34082b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f4648a24b80 1 --2- 172.21.15.22:0/1452307042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 0x562b34082b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f463808bbb0 tx=0x7f4630001360 comp rx=0 tx=0).stop 2023-07-06T01:21:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f4648a24b80 1 -- 172.21.15.22:0/1452307042 shutdown_connections 2023-07-06T01:21:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f4648a24b80 1 --2- 172.21.15.22:0/1452307042 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x562b340d0ae0 0x562b340830d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f4648a24b80 1 --2- 172.21.15.22:0/1452307042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562b34069d10 0x562b34082b90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f46157fa700 0 client.5349 ms_handle_reset on v2:172.21.15.22:6833/612827809 2023-07-06T01:21:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f4648a24b80 1 --2- 172.21.15.22:0/1452307042 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x562b3403f870 0x562b34068220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f4648a24b80 1 --2- 172.21.15.22:0/1452307042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562b340694b0 0x562b34068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f4648a24b80 1 -- 172.21.15.22:0/1452307042 >> 172.21.15.22:0/1452307042 conn(0x562b34036ec0 msgr2=0x562b3408af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f4648a24b80 1 -- 172.21.15.22:0/1452307042 shutdown_connections 2023-07-06T01:21:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.620+0000 7f4648a24b80 1 -- 172.21.15.22:0/1452307042 wait complete. 2023-07-06T01:21:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.621+0000 7f4648a24b80 20 client.5349 trim_cache size 0 max 16384 2023-07-06T01:21:35.639 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:21:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-0": { 2023-07-06T01:21:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 1997, 2023-07-06T01:21:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 1984, 2023-07-06T01:21:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 688976, 2023-07-06T01:21:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 21373545, 2023-07-06T01:21:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 8, 2023-07-06T01:21:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 3, 2023-07-06T01:21:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.227355240, 2023-07-06T01:21:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.124077277, 2023-07-06T01:21:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.036409460, 2023-07-06T01:21:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.065430837, 2023-07-06T01:21:35.640 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T01:21:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1983, 2023-07-06T01:21:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 2.976595715, 2023-07-06T01:21:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.001501056 2023-07-06T01:21:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-07-06T01:21:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 2, 2023-07-06T01:21:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000294, 2023-07-06T01:21:35.641 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000147 2023-07-06T01:21:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 126320, 2023-07-06T01:21:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 21616 2023-07-06T01:21:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-1": { 2023-07-06T01:21:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 2672, 2023-07-06T01:21:35.642 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 2688, 2023-07-06T01:21:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 802280, 2023-07-06T01:21:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 19176307, 2023-07-06T01:21:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 6, 2023-07-06T01:21:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 4, 2023-07-06T01:21:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 0.204656287, 2023-07-06T01:21:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 0.098522070, 2023-07-06T01:21:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 0.033135020, 2023-07-06T01:21:35.643 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.065631045, 2023-07-06T01:21:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T01:21:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 2687, 2023-07-06T01:21:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 1.972785955, 2023-07-06T01:21:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000734196 2023-07-06T01:21:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-07-06T01:21:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.644 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 1728, 2023-07-06T01:21:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 110256 2023-07-06T01:21:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "AsyncMessenger::Worker-2": { 2023-07-06T01:21:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_messages": 33018, 2023-07-06T01:21:35.645 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages": 56588, 2023-07-06T01:21:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_bytes": 10530853, 2023-07-06T01:21:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_bytes": 72880117, 2023-07-06T01:21:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_created_connections": 8, 2023-07-06T01:21:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_active_connections": 4, 2023-07-06T01:21:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_total_time": 2.330496746, 2023-07-06T01:21:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_send_time": 1.466947867, 2023-07-06T01:21:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_recv_time": 1.013135758, 2023-07-06T01:21:35.646 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_running_fast_dispatch_time": 0.108874025, 2023-07-06T01:21:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_messages_queue_lat": { 2023-07-06T01:21:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 56587, 2023-07-06T01:21:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 4.660135277, 2023-07-06T01:21:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000082353 2023-07-06T01:21:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.647 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_handle_ack_lat": { 2023-07-06T01:21:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 50419, 2023-07-06T01:21:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.010602755, 2023-07-06T01:21:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000210 2023-07-06T01:21:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-06T01:21:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: "msgr_send_encrypted_bytes": 0 2023-07-06T01:21:35.648 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "cct": { 2023-07-06T01:21:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "total_workers": 1, 2023-07-06T01:21:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "unhealthy_workers": 0 2023-07-06T01:21:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "finisher-MDSRank": { 2023-07-06T01:21:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "queue_len": 0, 2023-07-06T01:21:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "complete_latency": { 2023-07-06T01:21:35.649 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1051, 2023-07-06T01:21:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 19.326341939, 2023-07-06T01:21:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.018388527 2023-07-06T01:21:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:21:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "finisher-PurgeQueue": { 2023-07-06T01:21:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "queue_len": 0, 2023-07-06T01:21:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "complete_latency": { 2023-07-06T01:21:35.650 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 2228, 2023-07-06T01:21:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 1.276708458, 2023-07-06T01:21:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000573028 2023-07-06T01:21:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:21:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": { 2023-07-06T01:21:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "request": 28558, 2023-07-06T01:21:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "reply": 28558, 2023-07-06T01:21:35.651 INFO:teuthology.orchestra.run.smithi022.stdout: "reply_latency": { 2023-07-06T01:21:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 28558, 2023-07-06T01:21:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 662.856260458, 2023-07-06T01:21:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.023210878 2023-07-06T01:21:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "slow_reply": 0, 2023-07-06T01:21:35.652 INFO:teuthology.orchestra.run.smithi022.stdout: "forward": 0, 2023-07-06T01:21:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_fetch_complete": 0, 2023-07-06T01:21:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_fetch_keys": 0, 2023-07-06T01:21:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_commit": 36, 2023-07-06T01:21:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_split": 0, 2023-07-06T01:21:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_merge": 0, 2023-07-06T01:21:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes": 18230, 2023-07-06T01:21:35.653 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_top": 10258, 2023-07-06T01:21:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_bottom": 7972, 2023-07-06T01:21:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_pin_tail": 0, 2023-07-06T01:21:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_pinned": 3575, 2023-07-06T01:21:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_expired": 0, 2023-07-06T01:21:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "inodes_with_caps": 96, 2023-07-06T01:21:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": 96, 2023-07-06T01:21:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "subtrees": 2, 2023-07-06T01:21:35.654 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse": 118930, 2023-07-06T01:21:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_hit": 115992, 2023-07-06T01:21:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_forward": 0, 2023-07-06T01:21:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_discover": 0, 2023-07-06T01:21:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_dir_fetch": 0, 2023-07-06T01:21:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_remote_ino": 0, 2023-07-06T01:21:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "traverse_lock": 1893, 2023-07-06T01:21:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "load_cent": 653718, 2023-07-06T01:21:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: "q": 46, 2023-07-06T01:21:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "exported": 0, 2023-07-06T01:21:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "exported_inodes": 0, 2023-07-06T01:21:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "imported": 0, 2023-07-06T01:21:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "imported_inodes": 0, 2023-07-06T01:21:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_dir_fetch": 0, 2023-07-06T01:21:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_backtrace_fetch": 0, 2023-07-06T01:21:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "openino_peer_discover": 0, 2023-07-06T01:21:35.656 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rfiles": 0, 2023-07-06T01:21:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rbytes": 0, 2023-07-06T01:21:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "root_rsnaps": 0, 2023-07-06T01:21:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_backtrace_fetch": 0, 2023-07-06T01:21:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_set_tag": 0, 2023-07-06T01:21:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_backtrace_repaired": 0, 2023-07-06T01:21:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_inotable_repaired": 0, 2023-07-06T01:21:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dir_inodes": 0, 2023-07-06T01:21:35.657 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dir_base_inodes": 0, 2023-07-06T01:21:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_dirfrag_rstats": 0, 2023-07-06T01:21:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_file_inodes": 0, 2023-07-06T01:21:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_inode_file_caps": 0, 2023-07-06T01:21:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_revoke": 2327, 2023-07-06T01:21:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_grant": 5682, 2023-07-06T01:21:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_trunc": 0, 2023-07-06T01:21:35.658 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-06T01:21:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-06T01:21:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_caps": 4804, 2023-07-06T01:21:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_caps_dirty": 2478, 2023-07-06T01:21:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_cap_release": 60, 2023-07-06T01:21:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "process_request_cap_release": 13976 2023-07-06T01:21:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_cache": { 2023-07-06T01:21:35.659 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_update": 0, 2023-07-06T01:21:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_update_receipt": 0, 2023-07-06T01:21:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_try_discover": 0, 2023-07-06T01:21:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_send_discover": 0, 2023-07-06T01:21:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: "dir_handle_discover": 0, 2023-07-06T01:21:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays": 2000, 2023-07-06T01:21:35.660 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays_delayed": 0, 2023-07-06T01:21:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "num_strays_enqueuing": 0, 2023-07-06T01:21:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_created": 4221, 2023-07-06T01:21:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_enqueued": 1906, 2023-07-06T01:21:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_reintegrated": 4511, 2023-07-06T01:21:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "strays_migrated": 0, 2023-07-06T01:21:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_processing": 0, 2023-07-06T01:21:35.661 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_enqueued": 0, 2023-07-06T01:21:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "num_recovering_prioritized": 0, 2023-07-06T01:21:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_started": 0, 2023-07-06T01:21:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "recovery_completed": 0, 2023-07-06T01:21:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_enqueue_scrub": 0, 2023-07-06T01:21:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_exportdir": 0, 2023-07-06T01:21:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_flush": 0, 2023-07-06T01:21:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_fragmentdir": 0, 2023-07-06T01:21:35.662 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_fragstats": 0, 2023-07-06T01:21:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "ireq_inodestats": 0 2023-07-06T01:21:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_log": { 2023-07-06T01:21:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "evadd": 24687, 2023-07-06T01:21:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "evex": 20873, 2023-07-06T01:21:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "evtrm": 18228, 2023-07-06T01:21:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "ev": 6459, 2023-07-06T01:21:35.663 INFO:teuthology.orchestra.run.smithi022.stdout: "evexg": 0, 2023-07-06T01:21:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "evexd": 2645, 2023-07-06T01:21:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "segadd": 33, 2023-07-06T01:21:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "segex": 26, 2023-07-06T01:21:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "segtrm": 23, 2023-07-06T01:21:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "seg": 10, 2023-07-06T01:21:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "segexg": 0, 2023-07-06T01:21:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "segexd": 3, 2023-07-06T01:21:35.664 INFO:teuthology.orchestra.run.smithi022.stdout: "expos": 67114455, 2023-07-06T01:21:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: "wrpos": 93883692, 2023-07-06T01:21:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: "rdpos": 0, 2023-07-06T01:21:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: "jlat": { 2023-07-06T01:21:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 2667, 2023-07-06T01:21:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 96.173805667, 2023-07-06T01:21:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.036060669 2023-07-06T01:21:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.665 INFO:teuthology.orchestra.run.smithi022.stdout: "replayed": 0 2023-07-06T01:21:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_mem": { 2023-07-06T01:21:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 2014, 2023-07-06T01:21:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "ino+": 2014, 2023-07-06T01:21:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "ino-": 0, 2023-07-06T01:21:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "dir": 13, 2023-07-06T01:21:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "dir+": 13, 2023-07-06T01:21:35.666 INFO:teuthology.orchestra.run.smithi022.stdout: "dir-": 0, 2023-07-06T01:21:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "dn": 18230, 2023-07-06T01:21:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "dn+": 20022, 2023-07-06T01:21:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "dn-": 1792, 2023-07-06T01:21:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "cap": 96, 2023-07-06T01:21:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "cap+": 8414, 2023-07-06T01:21:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "cap-": 8318, 2023-07-06T01:21:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "rss": 128736, 2023-07-06T01:21:35.667 INFO:teuthology.orchestra.run.smithi022.stdout: "heap": 207196 2023-07-06T01:21:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_server": { 2023-07-06T01:21:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "dispatch_client_request": 65571, 2023-07-06T01:21:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "dispatch_server_request": 0, 2023-07-06T01:21:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_request": 28558, 2023-07-06T01:21:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_client_session": 4, 2023-07-06T01:21:35.668 INFO:teuthology.orchestra.run.smithi022.stdout: "handle_peer_request": 0, 2023-07-06T01:21:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: "req_create_latency": { 2023-07-06T01:21:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 2000, 2023-07-06T01:21:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 1.865850446, 2023-07-06T01:21:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000932925 2023-07-06T01:21:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.669 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getattr_latency": { 2023-07-06T01:21:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 51, 2023-07-06T01:21:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.009168250, 2023-07-06T01:21:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000179769 2023-07-06T01:21:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getfilelock_latency": { 2023-07-06T01:21:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.670 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: "req_link_latency": { 2023-07-06T01:21:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 6000, 2023-07-06T01:21:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 8.208395536, 2023-07-06T01:21:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.001368065 2023-07-06T01:21:35.671 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookup_latency": { 2023-07-06T01:21:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 7983, 2023-07-06T01:21:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 3.550568337, 2023-07-06T01:21:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000444766 2023-07-06T01:21:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookuphash_latency": { 2023-07-06T01:21:35.672 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupino_latency": { 2023-07-06T01:21:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.673 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupname_latency": { 2023-07-06T01:21:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.674 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupparent_latency": { 2023-07-06T01:21:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.675 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lookupsnap_latency": { 2023-07-06T01:21:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1, 2023-07-06T01:21:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000052910, 2023-07-06T01:21:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000052910 2023-07-06T01:21:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "req_lssnap_latency": { 2023-07-06T01:21:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.676 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.677 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.677 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.677 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mkdir_latency": { 2023-07-06T01:21:35.677 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1, 2023-07-06T01:21:35.677 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.003066959, 2023-07-06T01:21:35.677 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.003066959 2023-07-06T01:21:35.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.678 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mknod_latency": { 2023-07-06T01:21:35.678 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.678 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.678 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.678 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.679 INFO:teuthology.orchestra.run.smithi022.stdout: "req_mksnap_latency": { 2023-07-06T01:21:35.679 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1, 2023-07-06T01:21:35.679 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.378157804, 2023-07-06T01:21:35.679 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.378157804 2023-07-06T01:21:35.679 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.679 INFO:teuthology.orchestra.run.smithi022.stdout: "req_open_latency": { 2023-07-06T01:21:35.679 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: "req_readdir_latency": { 2023-07-06T01:21:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 8, 2023-07-06T01:21:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.121590603, 2023-07-06T01:21:35.680 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.015198825 2023-07-06T01:21:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rename_latency": { 2023-07-06T01:21:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 4511, 2023-07-06T01:21:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 619.143396190, 2023-07-06T01:21:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.137251916 2023-07-06T01:21:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "req_renamesnap_latency": { 2023-07-06T01:21:35.681 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "req_snapdiff_latency": { 2023-07-06T01:21:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.682 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmdir_latency": { 2023-07-06T01:21:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmsnap_latency": { 2023-07-06T01:21:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1, 2023-07-06T01:21:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.128398997, 2023-07-06T01:21:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.128398997 2023-07-06T01:21:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "req_rmxattr_latency": { 2023-07-06T01:21:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.684 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setattr_latency": { 2023-07-06T01:21:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 1, 2023-07-06T01:21:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000284290, 2023-07-06T01:21:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000284290 2023-07-06T01:21:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setdirlayout_latency": { 2023-07-06T01:21:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.685 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setfilelock_latency": { 2023-07-06T01:21:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.686 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setlayout_latency": { 2023-07-06T01:21:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "req_setxattr_latency": { 2023-07-06T01:21:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.687 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "req_symlink_latency": { 2023-07-06T01:21:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.688 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "req_unlink_latency": { 2023-07-06T01:21:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 8000, 2023-07-06T01:21:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 29.447330136, 2023-07-06T01:21:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.003680916 2023-07-06T01:21:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "cap_revoke_eviction": 0, 2023-07-06T01:21:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "cap_acquisition_throttle": 0, 2023-07-06T01:21:35.689 INFO:teuthology.orchestra.run.smithi022.stdout: "req_getvxattr_latency": { 2023-07-06T01:21:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:21:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_sessions": { 2023-07-06T01:21:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "session_count": 1, 2023-07-06T01:21:35.690 INFO:teuthology.orchestra.run.smithi022.stdout: "session_add": 1, 2023-07-06T01:21:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "session_remove": 0, 2023-07-06T01:21:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "sessions_open": 1, 2023-07-06T01:21:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "sessions_stale": 0, 2023-07-06T01:21:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "total_load": 17941, 2023-07-06T01:21:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "average_load": 17941, 2023-07-06T01:21:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: "avg_session_uptime": 54 2023-07-06T01:21:35.691 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mempool": { 2023-07-06T01:21:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_bytes": 0, 2023-07-06T01:21:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "bloom_filter_items": 0, 2023-07-06T01:21:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_bytes": 0, 2023-07-06T01:21:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_alloc_items": 0, 2023-07-06T01:21:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_bytes": 0, 2023-07-06T01:21:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_data_items": 0, 2023-07-06T01:21:35.692 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-06T01:21:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_onode_items": 0, 2023-07-06T01:21:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-06T01:21:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_meta_items": 0, 2023-07-06T01:21:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_bytes": 0, 2023-07-06T01:21:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_other_items": 0, 2023-07-06T01:21:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-06T01:21:35.693 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_cache_buffer_items": 0, 2023-07-06T01:21:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_bytes": 0, 2023-07-06T01:21:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_extent_items": 0, 2023-07-06T01:21:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_bytes": 0, 2023-07-06T01:21:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_blob_items": 0, 2023-07-06T01:21:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-06T01:21:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_shared_blob_items": 0, 2023-07-06T01:21:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-06T01:21:35.694 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_inline_bl_items": 0, 2023-07-06T01:21:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_bytes": 0, 2023-07-06T01:21:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_fsck_items": 0, 2023-07-06T01:21:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_bytes": 0, 2023-07-06T01:21:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_txc_items": 0, 2023-07-06T01:21:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-06T01:21:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_deferred_items": 0, 2023-07-06T01:21:35.695 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_bytes": 0, 2023-07-06T01:21:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "bluestore_writing_items": 0, 2023-07-06T01:21:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_bytes": 0, 2023-07-06T01:21:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_items": 0, 2023-07-06T01:21:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_bytes": 0, 2023-07-06T01:21:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_reader_items": 0, 2023-07-06T01:21:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_bytes": 0, 2023-07-06T01:21:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "bluefs_file_writer_items": 0, 2023-07-06T01:21:35.696 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_bytes": 16432175, 2023-07-06T01:21:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_anon_items": 6124, 2023-07-06T01:21:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_bytes": 440, 2023-07-06T01:21:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "buffer_meta_items": 5, 2023-07-06T01:21:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_bytes": 0, 2023-07-06T01:21:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_items": 0, 2023-07-06T01:21:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_bytes": 0, 2023-07-06T01:21:35.697 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_mapbl_items": 0, 2023-07-06T01:21:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_bytes": 0, 2023-07-06T01:21:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_pglog_items": 0, 2023-07-06T01:21:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_bytes": 6272, 2023-07-06T01:21:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_items": 80, 2023-07-06T01:21:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_bytes": 0, 2023-07-06T01:21:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_mapping_items": 0, 2023-07-06T01:21:35.698 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_bytes": 0, 2023-07-06T01:21:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "pgmap_items": 0, 2023-07-06T01:21:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_bytes": 19526052, 2023-07-06T01:21:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mds_co_items": 856174, 2023-07-06T01:21:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_bytes": 0, 2023-07-06T01:21:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_1_items": 0, 2023-07-06T01:21:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_bytes": 0, 2023-07-06T01:21:35.699 INFO:teuthology.orchestra.run.smithi022.stdout: "unittest_2_items": 0 2023-07-06T01:21:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "objecter": { 2023-07-06T01:21:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "op_active": 0, 2023-07-06T01:21:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "op_laggy": 0, 2023-07-06T01:21:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "op_send": 8621, 2023-07-06T01:21:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "op_send_bytes": 100256887, 2023-07-06T01:21:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "op_resend": 37, 2023-07-06T01:21:35.700 INFO:teuthology.orchestra.run.smithi022.stdout: "op_reply": 8621, 2023-07-06T01:21:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "op_latency": { 2023-07-06T01:21:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 8621, 2023-07-06T01:21:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 150.091912709, 2023-07-06T01:21:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.017410035 2023-07-06T01:21:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "op_inflight": 0, 2023-07-06T01:21:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "oplen_avg": { 2023-07-06T01:21:35.701 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 8621, 2023-07-06T01:21:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 16224 2023-07-06T01:21:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "op": 8621, 2023-07-06T01:21:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "op_r": 65, 2023-07-06T01:21:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "op_w": 8556, 2023-07-06T01:21:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "op_rmw": 0, 2023-07-06T01:21:35.702 INFO:teuthology.orchestra.run.smithi022.stdout: "op_pg": 0, 2023-07-06T01:21:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_stat": 23, 2023-07-06T01:21:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_create": 3758, 2023-07-06T01:21:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_read": 65, 2023-07-06T01:21:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_write": 2732, 2023-07-06T01:21:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_writefull": 53, 2023-07-06T01:21:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_writesame": 0, 2023-07-06T01:21:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_append": 0, 2023-07-06T01:21:35.703 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_zero": 1906, 2023-07-06T01:21:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_truncate": 0, 2023-07-06T01:21:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_delete": 48, 2023-07-06T01:21:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_mapext": 0, 2023-07-06T01:21:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_sparse_read": 0, 2023-07-06T01:21:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_clonerange": 0, 2023-07-06T01:21:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_getxattr": 0, 2023-07-06T01:21:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_setxattr": 7516, 2023-07-06T01:21:35.704 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_cmpxattr": 0, 2023-07-06T01:21:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_rmxattr": 0, 2023-07-06T01:21:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_resetxattrs": 0, 2023-07-06T01:21:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_call": 0, 2023-07-06T01:21:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_watch": 0, 2023-07-06T01:21:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_notify": 0, 2023-07-06T01:21:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_src_cmpxattr": 0, 2023-07-06T01:21:35.705 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_pgls": 0, 2023-07-06T01:21:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_pgls_filter": 0, 2023-07-06T01:21:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "osdop_other": 0, 2023-07-06T01:21:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_active": 0, 2023-07-06T01:21:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_send": 0, 2023-07-06T01:21:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_resend": 0, 2023-07-06T01:21:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "linger_ping": 0, 2023-07-06T01:21:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_active": 0, 2023-07-06T01:21:35.706 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_send": 0, 2023-07-06T01:21:35.707 INFO:teuthology.orchestra.run.smithi022.stdout: "poolop_resend": 0, 2023-07-06T01:21:35.707 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_active": 0, 2023-07-06T01:21:35.707 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_send": 0, 2023-07-06T01:21:35.707 INFO:teuthology.orchestra.run.smithi022.stdout: "poolstat_resend": 0, 2023-07-06T01:21:35.707 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_active": 0, 2023-07-06T01:21:35.707 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_send": 0, 2023-07-06T01:21:35.707 INFO:teuthology.orchestra.run.smithi022.stdout: "statfs_resend": 0, 2023-07-06T01:21:35.707 INFO:teuthology.orchestra.run.smithi022.stdout: "command_active": 0, 2023-07-06T01:21:35.708 INFO:teuthology.orchestra.run.smithi022.stdout: "command_send": 0, 2023-07-06T01:21:35.708 INFO:teuthology.orchestra.run.smithi022.stdout: "command_resend": 0, 2023-07-06T01:21:35.708 INFO:teuthology.orchestra.run.smithi022.stdout: "map_epoch": 36, 2023-07-06T01:21:35.708 INFO:teuthology.orchestra.run.smithi022.stdout: "map_full": 0, 2023-07-06T01:21:35.708 INFO:teuthology.orchestra.run.smithi022.stdout: "map_inc": 3, 2023-07-06T01:21:35.708 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_sessions": 8, 2023-07-06T01:21:35.708 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_session_open": 8, 2023-07-06T01:21:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_session_close": 0, 2023-07-06T01:21:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "osd_laggy": 0, 2023-07-06T01:21:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_wr": 95, 2023-07-06T01:21:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_rd": 0, 2023-07-06T01:21:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_del": 28 2023-07-06T01:21:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "oft": { 2023-07-06T01:21:35.709 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_objs": 1, 2023-07-06T01:21:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_kv_pairs": 0, 2023-07-06T01:21:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_updates": 0, 2023-07-06T01:21:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: "omap_total_removes": 0 2023-07-06T01:21:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: "osd-slow-ops": { 2023-07-06T01:21:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: "slow_ops_count": 0 2023-07-06T01:21:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.710 INFO:teuthology.orchestra.run.smithi022.stdout: "purge_queue": { 2023-07-06T01:21:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_ops": 0, 2023-07-06T01:21:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_ops_high_water": 32, 2023-07-06T01:21:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing": 0, 2023-07-06T01:21:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executing_high_water": 32, 2023-07-06T01:21:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_executed": 1906, 2023-07-06T01:21:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "pq_item_in_journal": 0 2023-07-06T01:21:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.711 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-06T01:21:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-07-06T01:21:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2023-07-06T01:21:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-07-06T01:21:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 37682, 2023-07-06T01:21:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 9088818, 2023-07-06T01:21:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-07-06T01:21:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 37682, 2023-07-06T01:21:35.712 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2023-07-06T01:21:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2023-07-06T01:21:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 37682, 2023-07-06T01:21:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 9088818, 2023-07-06T01:21:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-07-06T01:21:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.713 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:21:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-objecter_bytes": { 2023-07-06T01:21:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-07-06T01:21:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 104857600, 2023-07-06T01:21:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-07-06T01:21:35.714 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2023-07-06T01:21:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2023-07-06T01:21:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-07-06T01:21:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2023-07-06T01:21:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 8621, 2023-07-06T01:21:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 100464641, 2023-07-06T01:21:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 8621, 2023-07-06T01:21:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 100464641, 2023-07-06T01:21:35.715 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-07-06T01:21:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:21:35.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-objecter_ops": { 2023-07-06T01:21:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-07-06T01:21:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 1024, 2023-07-06T01:21:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-07-06T01:21:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 0, 2023-07-06T01:21:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 0, 2023-07-06T01:21:35.717 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-07-06T01:21:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 0, 2023-07-06T01:21:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 8621, 2023-07-06T01:21:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 8621, 2023-07-06T01:21:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 8621, 2023-07-06T01:21:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 8621, 2023-07-06T01:21:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-07-06T01:21:35.718 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:21:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-write_buf_throttle": { 2023-07-06T01:21:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 0, 2023-07-06T01:21:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 3758096384, 2023-07-06T01:21:35.719 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-07-06T01:21:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 1906, 2023-07-06T01:21:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 207754, 2023-07-06T01:21:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-07-06T01:21:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 1906, 2023-07-06T01:21:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2023-07-06T01:21:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2023-07-06T01:21:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 65, 2023-07-06T01:21:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 207754, 2023-07-06T01:21:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-07-06T01:21:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:21:35.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2023-07-06T01:21:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "throttle-write_buf_throttle-0x564610085cc0": { 2023-07-06T01:21:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "val": 84, 2023-07-06T01:21:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "max": 3758096384, 2023-07-06T01:21:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "get_started": 0, 2023-07-06T01:21:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "get": 24687, 2023-07-06T01:21:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "get_sum": 89689388, 2023-07-06T01:21:35.722 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_fail": 0, 2023-07-06T01:21:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "get_or_fail_success": 24687, 2023-07-06T01:21:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "take": 0, 2023-07-06T01:21:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "take_sum": 0, 2023-07-06T01:21:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "put": 2667, 2023-07-06T01:21:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "put_sum": 89689304, 2023-07-06T01:21:35.723 INFO:teuthology.orchestra.run.smithi022.stdout: "wait": { 2023-07-06T01:21:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: "avgcount": 0, 2023-07-06T01:21:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: "sum": 0.000000000, 2023-07-06T01:21:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: "avgtime": 0.000000000 2023-07-06T01:21:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:21:35.724 INFO:teuthology.orchestra.run.smithi022.stdout: } 2023-07-06T01:21:35.724 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:21:35.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.621+0000 7f463c89f700 1 -- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f461806ace0 msgr2=0x7f461806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.621+0000 7f463c89f700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f461806ace0 0x7f461806d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f462c0090b0 tx=0x7f462c008030 comp rx=0 tx=0).stop 2023-07-06T01:21:35.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.621+0000 7f463c89f700 1 -- 172.21.15.22:0/3306830851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46380c75f0 msgr2=0x7f46381f3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.621+0000 7f463c89f700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46380c75f0 0x7f46381f3520 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4628011aa0 tx=0x7f4628011db0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.621+0000 7f463c89f700 1 -- 172.21.15.22:0/3306830851 shutdown_connections 2023-07-06T01:21:35.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.621+0000 7f463c89f700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f46380c8030 0x7f46380d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.621+0000 7f463c89f700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f461806ace0 0x7f461806d190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.621+0000 7f463c89f700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46380cf5a0 0x7f463815fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.621+0000 7f463c89f700 1 --2- 172.21.15.22:0/3306830851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46380c75f0 0x7f46381f3520 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.621+0000 7f463c89f700 1 -- 172.21.15.22:0/3306830851 >> 172.21.15.22:0/3306830851 conn(0x7f463809c2e0 msgr2=0x7f463815c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:35.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.621+0000 7f463c89f700 1 -- 172.21.15.22:0/3306830851 shutdown_connections 2023-07-06T01:21:35.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.621+0000 7f463c89f700 1 -- 172.21.15.22:0/3306830851 wait complete. 2023-07-06T01:21:35.728 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2023-07-06T01:21:35.729 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub start / scrub_mdsdir 2023-07-06T01:21:35.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.857+0000 7f3c5b8e6700 1 Processor -- start 2023-07-06T01:21:35.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.857+0000 7f3c5b8e6700 1 -- start start 2023-07-06T01:21:35.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.858+0000 7f3c5b8e6700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3c5406b010 0x7f3c54164360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.858+0000 7f3c5b8e6700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c540cd640 con 0x7f3c5415a690 2023-07-06T01:21:35.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.858+0000 7f3c5b8e6700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c540cd920 con 0x7f3c540c9a40 2023-07-06T01:21:35.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.858+0000 7f3c5b8e6700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c540cdc00 con 0x7f3c5406b010 2023-07-06T01:21:35.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.858+0000 7f3c58e81700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f3c5415a690 0x7f3c540ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38428/0 (socket says 172.21.15.22:38428) 2023-07-06T01:21:35.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.858+0000 7f3c58e81700 1 -- 172.21.15.22:0/4207411422 learned_addr learned my addr 172.21.15.22:0/4207411422 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:35.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.859+0000 7f3c59e83700 1 --2- 172.21.15.22:0/4207411422 >> v2:172.21.15.103:3300/0 conn(0x7f3c5406b010 0x7f3c54164360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.859+0000 7f3c4bfff700 1 -- 172.21.15.22:0/4207411422 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282882359 0 0) 0x7f3c540cd640 con 0x7f3c5415a690 2023-07-06T01:21:35.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.859+0000 7f3c4bfff700 1 -- 172.21.15.22:0/4207411422 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c400035b0 con 0x7f3c5415a690 2023-07-06T01:21:35.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.859+0000 7f3c59e83700 1 -- 172.21.15.22:0/4207411422 >> v1:172.21.15.103:6789/0 conn(0x7f3c540c9a40 legacy=0x7f3c540c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.859+0000 7f3c59e83700 1 -- 172.21.15.22:0/4207411422 >> v1:172.21.15.22:6789/0 conn(0x7f3c5415a690 legacy=0x7f3c540ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.859+0000 7f3c4bfff700 1 -- 172.21.15.22:0/4207411422 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1274336728 0 0) 0x7f3c400035b0 con 0x7f3c5415a690 2023-07-06T01:21:35.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.859+0000 7f3c59e83700 1 -- 172.21.15.22:0/4207411422 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c540cdee0 con 0x7f3c5406b010 2023-07-06T01:21:35.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.859+0000 7f3c59e83700 1 --2- 172.21.15.22:0/4207411422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c5406b010 0x7f3c54164360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009d90 tx=0x7f3c4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e775e3e7bbb716f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.859+0000 7f3c4bfff700 1 -- 172.21.15.22:0/4207411422 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00e070 con 0x7f3c5406b010 2023-07-06T01:21:35.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.859+0000 7f3c4bfff700 1 -- 172.21.15.22:0/4207411422 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c003a20 con 0x7f3c5406b010 2023-07-06T01:21:35.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.859+0000 7f3c4bfff700 1 -- 172.21.15.22:0/4207411422 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50001670 con 0x7f3c5406b010 2023-07-06T01:21:35.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.860+0000 7f3c5b8e6700 1 -- 172.21.15.22:0/4207411422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c5406b010 msgr2=0x7f3c54164360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.860+0000 7f3c5b8e6700 1 --2- 172.21.15.22:0/4207411422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c5406b010 0x7f3c54164360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009d90 tx=0x7f3c4c00b330 comp rx=0 tx=0).stop 2023-07-06T01:21:35.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.860+0000 7f3c5b8e6700 1 -- 172.21.15.22:0/4207411422 shutdown_connections 2023-07-06T01:21:35.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.860+0000 7f3c5b8e6700 1 --2- 172.21.15.22:0/4207411422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c5406b010 0x7f3c54164360 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.860+0000 7f3c5b8e6700 1 -- 172.21.15.22:0/4207411422 >> 172.21.15.22:0/4207411422 conn(0x7f3c540ccb00 msgr2=0x7f3c540cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:35.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.860+0000 7f3c5b8e6700 1 -- 172.21.15.22:0/4207411422 shutdown_connections 2023-07-06T01:21:35.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.860+0000 7f3c5b8e6700 1 -- 172.21.15.22:0/4207411422 wait complete. 2023-07-06T01:21:35.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.861+0000 7f3c5b8e6700 1 Processor -- start 2023-07-06T01:21:35.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.861+0000 7f3c5b8e6700 1 -- start start 2023-07-06T01:21:35.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c5b8e6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3c5406b010 0x7f3c54146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c5b8e6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c540c9a40 0x7f3c5414cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c5b8e6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c5415a690 0x7f3c54149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c5b8e6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c541472e0 con 0x7f3c5415a690 2023-07-06T01:21:35.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c5b8e6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54146fd0 con 0x7f3c540c9a40 2023-07-06T01:21:35.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c5b8e6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c540c73d0 con 0x7f3c5406b010 2023-07-06T01:21:35.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c59e83700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c5415a690 0x7f3c54149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c59e83700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c5415a690 0x7f3c54149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47694/0 (socket says 172.21.15.22:47694) 2023-07-06T01:21:35.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c59e83700 1 -- 172.21.15.22:0/80933850 learned_addr learned my addr 172.21.15.22:0/80933850 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:35.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c58e81700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c540c9a40 0x7f3c5414cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c59682700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3c5406b010 0x7f3c54146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c59e83700 1 -- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3c5406b010 msgr2=0x7f3c54146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c59e83700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3c5406b010 0x7f3c54146a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c59e83700 1 -- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c540c9a40 msgr2=0x7f3c5414cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c59e83700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c540c9a40 0x7f3c5414cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c59e83700 1 -- 172.21.15.22:0/80933850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c009a70 con 0x7f3c5415a690 2023-07-06T01:21:35.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.862+0000 7f3c59682700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3c5406b010 0x7f3c54146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:35.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.863+0000 7f3c59e83700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c5415a690 0x7f3c54149370 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3c5406d3a0 tx=0x7f3c4c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.863+0000 7f3c49ffb700 1 -- 172.21.15.22:0/80933850 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c004440 con 0x7f3c5415a690 2023-07-06T01:21:35.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.863+0000 7f3c49ffb700 1 -- 172.21.15.22:0/80933850 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c0045e0 con 0x7f3c5415a690 2023-07-06T01:21:35.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.863+0000 7f3c5b8e6700 1 -- 172.21.15.22:0/80933850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c540c76b0 con 0x7f3c5415a690 2023-07-06T01:21:35.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.863+0000 7f3c49ffb700 1 -- 172.21.15.22:0/80933850 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c01c460 con 0x7f3c5415a690 2023-07-06T01:21:35.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.863+0000 7f3c5b8e6700 1 -- 172.21.15.22:0/80933850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c540c7c70 con 0x7f3c5415a690 2023-07-06T01:21:35.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.864+0000 7f3c49ffb700 1 -- 172.21.15.22:0/80933850 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3c4c01c5c0 con 0x7f3c5415a690 2023-07-06T01:21:35.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.865+0000 7f3c49ffb700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3c3806ace0 0x7f3c3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.865+0000 7f3c49ffb700 1 -- 172.21.15.22:0/80933850 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f3c4c08ac40 con 0x7f3c5415a690 2023-07-06T01:21:35.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.865+0000 7f3c59682700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3c3806ace0 0x7f3c3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.866+0000 7f3c59682700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3c3806ace0 0x7f3c3806d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3c44000f60 tx=0x7f3c44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.867+0000 7f3c67a6bb80 1 Processor -- start 2023-07-06T01:21:35.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.867+0000 7f3c67a6bb80 1 -- start start 2023-07-06T01:21:35.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c67a6bb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 0x55a1ad52d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c67a6bb80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad4e48f0 0x55a1ad5112b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c67a6bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad50ed90 0x55a1ad5117f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.876 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c67a6bb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ad3ebd20 con 0x55a1ad50e530 2023-07-06T01:21:35.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c67a6bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ad43eab0 con 0x55a1ad50ed90 2023-07-06T01:21:35.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c67a6bb80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1ad437f40 con 0x55a1ad4e48f0 2023-07-06T01:21:35.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c59682700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 0x55a1ad52d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c59682700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 0x55a1ad52d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47698/0 (socket says 172.21.15.22:47698) 2023-07-06T01:21:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c59682700 1 -- 172.21.15.22:0/273435098 learned_addr learned my addr 172.21.15.22:0/273435098 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c58e81700 1 --2- 172.21.15.22:0/273435098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad4e48f0 0x55a1ad5112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c59e83700 1 --2- 172.21.15.22:0/273435098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad50ed90 0x55a1ad5117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c59682700 1 -- 172.21.15.22:0/273435098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad4e48f0 msgr2=0x55a1ad5112b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c59682700 1 --2- 172.21.15.22:0/273435098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad4e48f0 0x55a1ad5112b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c59682700 1 -- 172.21.15.22:0/273435098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad50ed90 msgr2=0x55a1ad5117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c59682700 1 --2- 172.21.15.22:0/273435098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad50ed90 0x55a1ad5117f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c59682700 1 -- 172.21.15.22:0/273435098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1ad265e50 con 0x55a1ad50e530 2023-07-06T01:21:35.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c58e81700 1 --2- 172.21.15.22:0/273435098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad4e48f0 0x55a1ad5112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:35.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c59682700 1 --2- 172.21.15.22:0/273435098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 0x55a1ad52d730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3c440147f0 tx=0x7f3c44010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33ac075df7bcfbd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c3effd700 1 -- 172.21.15.22:0/273435098 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44012860 con 0x55a1ad50e530 2023-07-06T01:21:35.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c3effd700 1 -- 172.21.15.22:0/273435098 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c440129c0 con 0x55a1ad50e530 2023-07-06T01:21:35.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.868+0000 7f3c3effd700 1 -- 172.21.15.22:0/273435098 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44012d30 con 0x55a1ad50e530 2023-07-06T01:21:35.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.869+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/273435098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 msgr2=0x55a1ad52d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.869+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/273435098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 0x55a1ad52d730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3c440147f0 tx=0x7f3c44010000 comp rx=0 tx=0).stop 2023-07-06T01:21:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.869+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/273435098 shutdown_connections 2023-07-06T01:21:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.869+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/273435098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad50ed90 0x55a1ad5117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.869+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/273435098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 0x55a1ad52d730 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.869+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/273435098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad4e48f0 0x55a1ad5112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.869+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/273435098 >> 172.21.15.22:0/273435098 conn(0x55a1ad4dbf40 msgr2=0x55a1ad52c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.869+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/273435098 shutdown_connections 2023-07-06T01:21:35.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.869+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/273435098 wait complete. 2023-07-06T01:21:35.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.870+0000 7f3c67a6bb80 1 -- start start 2023-07-06T01:21:35.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.870+0000 7f3c67a6bb80 1 Processor -- start 2023-07-06T01:21:35.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c67a6bb80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad4e48f0 0x55a1ad509e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c67a6bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad50e530 0x55a1ad523360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c67a6bb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 0x55a1ad511530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c67a6bb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ad3d7b30 con 0x55a1ad50ed90 2023-07-06T01:21:35.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c67a6bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ad3d7830 con 0x55a1ad50e530 2023-07-06T01:21:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c67a6bb80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1ad2bb990 con 0x55a1ad4e48f0 2023-07-06T01:21:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c67a6bb80 10 client.0 _ll_register_callbacks cb 0x55a1ad3d57d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T01:21:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c59e83700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 0x55a1ad511530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c59e83700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 0x55a1ad511530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47708/0 (socket says 172.21.15.22:47708) 2023-07-06T01:21:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c59e83700 1 -- 172.21.15.22:0/3844185909 learned_addr learned my addr 172.21.15.22:0/3844185909 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c58e81700 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad50e530 0x55a1ad523360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c59682700 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad4e48f0 0x55a1ad509e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c59e83700 1 -- 172.21.15.22:0/3844185909 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad4e48f0 msgr2=0x55a1ad509e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c59e83700 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad4e48f0 0x55a1ad509e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c59e83700 1 -- 172.21.15.22:0/3844185909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad50e530 msgr2=0x55a1ad523360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c59e83700 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad50e530 0x55a1ad523360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c59e83700 1 -- 172.21.15.22:0/3844185909 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c540cd920 con 0x55a1ad50ed90 2023-07-06T01:21:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c59682700 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad4e48f0 0x55a1ad509e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c59e83700 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 0x55a1ad511530 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c055e80 tx=0x7f3c4c061cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde51e6a42ce6c53 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c33fff700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-07-06T01:21:35.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c33fff700 1 -- 172.21.15.22:0/3844185909 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c058e70 con 0x55a1ad50ed90 2023-07-06T01:21:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c33fff700 1 -- 172.21.15.22:0/3844185909 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f3c4c06edb0 con 0x55a1ad50ed90 2023-07-06T01:21:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.871+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/3844185909 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1ad4d9550 con 0x55a1ad50ed90 2023-07-06T01:21:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c33fff700 1 -- 172.21.15.22:0/3844185909 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c4c058fd0 con 0x55a1ad50ed90 2023-07-06T01:21:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c67a6bb80 10 client.5361 fetch_fsmap learned FSMap version 23 2023-07-06T01:21:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/3844185909 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55a1ad24ecd0 con 0x55a1ad50ed90 2023-07-06T01:21:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c33fff700 1 -- 172.21.15.22:0/3844185909 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a1ad4d9550 con 0x55a1ad50ed90 2023-07-06T01:21:35.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c33fff700 1 -- 172.21.15.22:0/3844185909 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f3c4c061840 con 0x55a1ad50ed90 2023-07-06T01:21:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c67a6bb80 10 client.5361 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T01:21:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c67a6bb80 10 client.5361 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-07-06T01:21:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c67a6bb80 20 client.5361 populate_metadata read hostname 'smithi022' 2023-07-06T01:21:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x55a1ad50ab30 0x55a1ad50af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c67a6bb80 4 client.5361 mds_command: new command op to 5115 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T01:21:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/3844185909 --> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1ad430f20 con 0x55a1ad50ab30 2023-07-06T01:21:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c59682700 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x55a1ad50ab30 0x55a1ad50af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c33fff700 10 client.5361 ms_handle_connect on v2:172.21.15.22:6833/612827809 2023-07-06T01:21:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.872+0000 7f3c59682700 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x55a1ad50ab30 0x55a1ad50af30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=631b97fbc4c255c server_cookie=27824e5b19c3e321 in_seq=0 out_seq=0 2023-07-06T01:21:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c33fff700 1 -- 172.21.15.22:0/3844185909 <== mds.0 v2:172.21.15.22:6833/612827809 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a1ad430f20 con 0x55a1ad50ab30 2023-07-06T01:21:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c33fff700 10 client.5361 handle_command_reply: tid=0 2023-07-06T01:21:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c67a6bb80 1 client.5361 shutdown 2023-07-06T01:21:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/3844185909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 msgr2=0x55a1ad511530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 0x55a1ad511530 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c055e80 tx=0x7f3c4c061cd0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.886 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/3844185909 shutdown_connections 2023-07-06T01:21:35.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x55a1ad50ab30 0x55a1ad50af30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 0x55a1ad511530 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad4e48f0 0x55a1ad509e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/3844185909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad50e530 0x55a1ad523360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c33fff700 0 client.5361 ms_handle_reset on v2:172.21.15.22:6833/612827809 2023-07-06T01:21:35.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/3844185909 >> 172.21.15.22:0/3844185909 conn(0x55a1ad4dbf40 msgr2=0x55a1ad4e1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:35.887 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/3844185909 shutdown_connections 2023-07-06T01:21:35.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/3844185909 wait complete. 2023-07-06T01:21:35.888 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.873+0000 7f3c67a6bb80 20 client.5361 trim_cache size 0 max 16384 2023-07-06T01:21:35.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.891+0000 7f3c67a6bb80 1 Processor -- start 2023-07-06T01:21:35.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.891+0000 7f3c67a6bb80 1 -- start start 2023-07-06T01:21:35.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c67a6bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad4e48f0 0x55a1ad509ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c67a6bb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 0x55a1ad58edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c67a6bb80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad50ed90 0x55a1ad4e0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c67a6bb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ad4dbae0 con 0x55a1ad50e530 2023-07-06T01:21:35.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c67a6bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ad512fc0 con 0x55a1ad4e48f0 2023-07-06T01:21:35.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c67a6bb80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1ad525c50 con 0x55a1ad50ed90 2023-07-06T01:21:35.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c58e81700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 0x55a1ad58edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c58e81700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 0x55a1ad58edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47714/0 (socket says 172.21.15.22:47714) 2023-07-06T01:21:35.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c58e81700 1 -- 172.21.15.22:0/2701247807 learned_addr learned my addr 172.21.15.22:0/2701247807 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:35.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c59682700 1 --2- 172.21.15.22:0/2701247807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad4e48f0 0x55a1ad509ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c59e83700 1 --2- 172.21.15.22:0/2701247807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad50ed90 0x55a1ad4e0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c58e81700 1 -- 172.21.15.22:0/2701247807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad50ed90 msgr2=0x55a1ad4e0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c58e81700 1 --2- 172.21.15.22:0/2701247807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad50ed90 0x55a1ad4e0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c58e81700 1 -- 172.21.15.22:0/2701247807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad4e48f0 msgr2=0x55a1ad509ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c58e81700 1 --2- 172.21.15.22:0/2701247807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad4e48f0 0x55a1ad509ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c58e81700 1 -- 172.21.15.22:0/2701247807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1ad28c2c0 con 0x55a1ad50e530 2023-07-06T01:21:35.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c59e83700 1 --2- 172.21.15.22:0/2701247807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad50ed90 0x55a1ad4e0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:35.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.892+0000 7f3c58e81700 1 --2- 172.21.15.22:0/2701247807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 0x55a1ad58edd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3c500011e0 tx=0x7f3c50005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45c409f57bf12cd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.893+0000 7f3c3f7fe700 1 -- 172.21.15.22:0/2701247807 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c500137a0 con 0x55a1ad50e530 2023-07-06T01:21:35.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.893+0000 7f3c3f7fe700 1 -- 172.21.15.22:0/2701247807 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5000d050 con 0x55a1ad50e530 2023-07-06T01:21:35.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.893+0000 7f3c3f7fe700 1 -- 172.21.15.22:0/2701247807 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50013b40 con 0x55a1ad50e530 2023-07-06T01:21:35.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.893+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/2701247807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 msgr2=0x55a1ad58edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.893+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/2701247807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 0x55a1ad58edd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3c500011e0 tx=0x7f3c50005b60 comp rx=0 tx=0).stop 2023-07-06T01:21:35.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.893+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/2701247807 shutdown_connections 2023-07-06T01:21:35.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.893+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/2701247807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad50ed90 0x55a1ad4e0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.893+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/2701247807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad4e48f0 0x55a1ad509ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.893+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/2701247807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50e530 0x55a1ad58edd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.893+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/2701247807 >> 172.21.15.22:0/2701247807 conn(0x55a1ad4dbf40 msgr2=0x55a1ad514a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:35.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.893+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/2701247807 shutdown_connections 2023-07-06T01:21:35.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.893+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/2701247807 wait complete. 2023-07-06T01:21:35.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.894+0000 7f3c67a6bb80 1 -- start start 2023-07-06T01:21:35.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.894+0000 7f3c67a6bb80 1 Processor -- start 2023-07-06T01:21:35.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.895+0000 7f3c67a6bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad4e48f0 0x55a1ad52ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.895+0000 7f3c67a6bb80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad50e530 0x55a1ad52d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.895+0000 7f3c67a6bb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 0x55a1ad57be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.895+0000 7f3c67a6bb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ad58f3c0 con 0x55a1ad50ed90 2023-07-06T01:21:35.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.895+0000 7f3c67a6bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ad5138c0 con 0x55a1ad4e48f0 2023-07-06T01:21:35.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.895+0000 7f3c67a6bb80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1ad4e11f0 con 0x55a1ad50e530 2023-07-06T01:21:35.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.895+0000 7f3c67a6bb80 10 client.0 _ll_register_callbacks cb 0x7f3c4c059570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T01:21:35.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c59e83700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 0x55a1ad57be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c59e83700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 0x55a1ad57be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47722/0 (socket says 172.21.15.22:47722) 2023-07-06T01:21:35.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c59e83700 1 -- 172.21.15.22:0/483620593 learned_addr learned my addr 172.21.15.22:0/483620593 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:35.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c58e81700 1 --2- 172.21.15.22:0/483620593 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad50e530 0x55a1ad52d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c59682700 1 --2- 172.21.15.22:0/483620593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad4e48f0 0x55a1ad52ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c59e83700 1 -- 172.21.15.22:0/483620593 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad50e530 msgr2=0x55a1ad52d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c59e83700 1 --2- 172.21.15.22:0/483620593 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad50e530 0x55a1ad52d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c59e83700 1 -- 172.21.15.22:0/483620593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad4e48f0 msgr2=0x55a1ad52ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:35.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c59e83700 1 --2- 172.21.15.22:0/483620593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad4e48f0 0x55a1ad52ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c59e83700 1 -- 172.21.15.22:0/483620593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c500014b0 con 0x55a1ad50ed90 2023-07-06T01:21:35.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c59e83700 1 --2- 172.21.15.22:0/483620593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 0x55a1ad57be90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0617a0 tx=0x7f3c4c088eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2677758fbfb114bc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:35.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c33fff700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-07-06T01:21:35.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c33fff700 1 -- 172.21.15.22:0/483620593 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c07c070 con 0x55a1ad50ed90 2023-07-06T01:21:35.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c33fff700 1 -- 172.21.15.22:0/483620593 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f3c4c061840 con 0x55a1ad50ed90 2023-07-06T01:21:35.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/483620593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c440129c0 con 0x55a1ad50ed90 2023-07-06T01:21:35.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c33fff700 1 -- 172.21.15.22:0/483620593 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c4c059750 con 0x55a1ad50ed90 2023-07-06T01:21:35.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c67a6bb80 10 client.5367 fetch_fsmap learned FSMap version 23 2023-07-06T01:21:35.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/483620593 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55a1ad264320 con 0x55a1ad50ed90 2023-07-06T01:21:35.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.896+0000 7f3c33fff700 1 -- 172.21.15.22:0/483620593 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c440129c0 con 0x55a1ad50ed90 2023-07-06T01:21:35.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.897+0000 7f3c33fff700 1 -- 172.21.15.22:0/483620593 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f3c4c07f020 con 0x55a1ad50ed90 2023-07-06T01:21:35.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.897+0000 7f3c67a6bb80 10 client.5367 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T01:21:35.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.897+0000 7f3c67a6bb80 10 client.5367 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-07-06T01:21:35.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.897+0000 7f3c67a6bb80 20 client.5367 populate_metadata read hostname 'smithi022' 2023-07-06T01:21:35.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.897+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/483620593 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x55a1ad52e140 0x55a1ad4e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:35.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.897+0000 7f3c67a6bb80 4 client.5367 mds_command: new command op to 5115 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["scrub_mdsdir"]}] 2023-07-06T01:21:35.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.897+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/483620593 --> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["scrub_mdsdir"]}) v1 -- 0x55a1ad3f30c0 con 0x55a1ad52e140 2023-07-06T01:21:35.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.897+0000 7f3c59682700 1 --2- 172.21.15.22:0/483620593 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x55a1ad52e140 0x55a1ad4e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:35.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.897+0000 7f3c33fff700 10 client.5367 ms_handle_connect on v2:172.21.15.22:6833/612827809 2023-07-06T01:21:35.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.897+0000 7f3c59682700 1 --2- 172.21.15.22:0/483620593 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x55a1ad52e140 0x55a1ad4e3380 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ae220a1dc11a83d server_cookie=fad820d79b4bd9e1 in_seq=0 out_seq=0 2023-07-06T01:21:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.909+0000 7f3c33fff700 1 -- 172.21.15.22:0/483620593 <== mds.0 v2:172.21.15.22:6833/612827809 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a1ad3f30c0 con 0x55a1ad52e140 2023-07-06T01:21:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.909+0000 7f3c33fff700 10 client.5367 handle_command_reply: tid=0 2023-07-06T01:21:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.910+0000 7f3c67a6bb80 1 client.5367 shutdown 2023-07-06T01:21:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.910+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/483620593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 msgr2=0x55a1ad57be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.910+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/483620593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 0x55a1ad57be90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0617a0 tx=0x7f3c4c088eb0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.910+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/483620593 shutdown_connections 2023-07-06T01:21:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.910+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/483620593 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x55a1ad52e140 0x55a1ad4e3380 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.910+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/483620593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a1ad50ed90 0x55a1ad57be90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.910+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/483620593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a1ad4e48f0 0x55a1ad52ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.910+0000 7f3c67a6bb80 1 --2- 172.21.15.22:0/483620593 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55a1ad50e530 0x55a1ad52d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.910+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/483620593 >> 172.21.15.22:0/483620593 conn(0x55a1ad4dbf40 msgr2=0x55a1ad514a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:35.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.910+0000 7f3c33fff700 0 client.5367 ms_handle_reset on v2:172.21.15.22:6833/612827809 2023-07-06T01:21:35.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.910+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/483620593 shutdown_connections 2023-07-06T01:21:35.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.910+0000 7f3c67a6bb80 1 -- 172.21.15.22:0/483620593 wait complete. 2023-07-06T01:21:35.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.910+0000 7f3c67a6bb80 20 client.5367 trim_cache size 0 max 16384 2023-07-06T01:21:35.914 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:21:35.915 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2023-07-06T01:21:35.915 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "3df72095-535c-4952-aa75-8713e98e5179", 2023-07-06T01:21:35.915 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2023-07-06T01:21:35.915 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:21:35.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.911+0000 7f3c5b8e6700 1 -- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3c3806ace0 msgr2=0x7f3c3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.911+0000 7f3c5b8e6700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3c3806ace0 0x7f3c3806d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3c44000f60 tx=0x7f3c44009040 comp rx=0 tx=0).stop 2023-07-06T01:21:35.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.911+0000 7f3c5b8e6700 1 -- 172.21.15.22:0/80933850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c5415a690 msgr2=0x7f3c54149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:35.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.911+0000 7f3c5b8e6700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c5415a690 0x7f3c54149370 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3c5406d3a0 tx=0x7f3c4c004ad0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.911+0000 7f3c5b8e6700 1 -- 172.21.15.22:0/80933850 shutdown_connections 2023-07-06T01:21:35.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.911+0000 7f3c5b8e6700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3c5415a690 0x7f3c54149370 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.911+0000 7f3c5b8e6700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3c5406b010 0x7f3c54146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.911+0000 7f3c5b8e6700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f3c3806ace0 0x7f3c3806d190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.911+0000 7f3c5b8e6700 1 --2- 172.21.15.22:0/80933850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c540c9a40 0x7f3c5414cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:35.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.911+0000 7f3c5b8e6700 1 -- 172.21.15.22:0/80933850 >> 172.21.15.22:0/80933850 conn(0x7f3c540ccb00 msgr2=0x7f3c540aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:35.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.911+0000 7f3c5b8e6700 1 -- 172.21.15.22:0/80933850 shutdown_connections 2023-07-06T01:21:35.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:35.911+0000 7f3c5b8e6700 1 -- 172.21.15.22:0/80933850 wait complete. 2023-07-06T01:21:35.929 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-07-06T01:21:36.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.114+0000 7f15fe4b7700 1 Processor -- start 2023-07-06T01:21:36.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.114+0000 7f15fe4b7700 1 -- start start 2023-07-06T01:21:36.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.115+0000 7f15fe4b7700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f15f80ccd70 0x7f15f815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.115+0000 7f15fe4b7700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f15f806d040 0x7f15f806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.115+0000 7f15fe4b7700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f80cf230 con 0x7f15f8159360 2023-07-06T01:21:36.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.115+0000 7f15fe4b7700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f815df20 con 0x7f15f806d040 2023-07-06T01:21:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.115+0000 7f15fe4b7700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f80ce620 con 0x7f15f80ccd70 2023-07-06T01:21:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.115+0000 7f15fca54700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f15f806d040 0x7f15f806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.115+0000 7f15f77fe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f15f80ccd70 0x7f15f815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.115+0000 7f15fca54700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f15f806d040 0x7f15f806d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47730/0 (socket says 172.21.15.22:47730) 2023-07-06T01:21:36.118 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.115+0000 7f15fca54700 1 -- 172.21.15.22:0/3317669522 learned_addr learned my addr 172.21.15.22:0/3317669522 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:36.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.116+0000 7f15fca54700 1 -- 172.21.15.22:0/3317669522 >> v2:172.21.15.103:3300/0 conn(0x7f15f80ccd70 msgr2=0x7f15f815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.116+0000 7f15fca54700 1 --2- 172.21.15.22:0/3317669522 >> v2:172.21.15.103:3300/0 conn(0x7f15f80ccd70 0x7f15f815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.116+0000 7f15fca54700 1 -- 172.21.15.22:0/3317669522 >> v1:172.21.15.103:6789/0 conn(0x7f15f8159360 legacy=0x7f15f80cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.116+0000 7f15fca54700 1 -- 172.21.15.22:0/3317669522 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f80ce900 con 0x7f15f806d040 2023-07-06T01:21:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.116+0000 7f15fca54700 1 --2- 172.21.15.22:0/3317669522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f806d040 0x7f15f806d4a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f15f80af7d0 tx=0x7f15f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf6285d95d18d99 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.116+0000 7f15f6ffd700 1 -- 172.21.15.22:0/3317669522 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f000e070 con 0x7f15f806d040 2023-07-06T01:21:36.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.116+0000 7f15f6ffd700 1 -- 172.21.15.22:0/3317669522 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f000b9c0 con 0x7f15f806d040 2023-07-06T01:21:36.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.116+0000 7f15f6ffd700 1 -- 172.21.15.22:0/3317669522 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f000bd70 con 0x7f15f806d040 2023-07-06T01:21:36.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.117+0000 7f15fe4b7700 1 -- 172.21.15.22:0/3317669522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f806d040 msgr2=0x7f15f806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.117+0000 7f15fe4b7700 1 --2- 172.21.15.22:0/3317669522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f806d040 0x7f15f806d4a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f15f80af7d0 tx=0x7f15f000b330 comp rx=0 tx=0).stop 2023-07-06T01:21:36.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.117+0000 7f15fe4b7700 1 -- 172.21.15.22:0/3317669522 shutdown_connections 2023-07-06T01:21:36.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.117+0000 7f15fe4b7700 1 --2- 172.21.15.22:0/3317669522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f806d040 0x7f15f806d4a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.117+0000 7f15fe4b7700 1 --2- 172.21.15.22:0/3317669522 >> v2:172.21.15.103:3300/0 conn(0x7f15f80ccd70 0x7f15f815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.117+0000 7f15fe4b7700 1 -- 172.21.15.22:0/3317669522 >> 172.21.15.22:0/3317669522 conn(0x7f15f80b1890 msgr2=0x7f15f80b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:36.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.117+0000 7f15fe4b7700 1 -- 172.21.15.22:0/3317669522 shutdown_connections 2023-07-06T01:21:36.122 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.117+0000 7f15fe4b7700 1 -- 172.21.15.22:0/3317669522 wait complete. 2023-07-06T01:21:36.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.119+0000 7f15fe4b7700 1 Processor -- start 2023-07-06T01:21:36.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.120+0000 7f15fe4b7700 1 -- start start 2023-07-06T01:21:36.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.120+0000 7f15fe4b7700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15f806d040 0x7f15f8154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.120+0000 7f15fe4b7700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15f80ccd70 0x7f15f80c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15fe4b7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f8159360 0x7f15f80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15fe4b7700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8154da0 con 0x7f15f8159360 2023-07-06T01:21:36.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15fe4b7700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f80c8720 con 0x7f15f806d040 2023-07-06T01:21:36.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15fe4b7700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f80c89c0 con 0x7f15f80ccd70 2023-07-06T01:21:36.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15fca54700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f8159360 0x7f15f80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.125 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15fca54700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f8159360 0x7f15f80b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47734/0 (socket says 172.21.15.22:47734) 2023-07-06T01:21:36.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15fca54700 1 -- 172.21.15.22:0/2185637408 learned_addr learned my addr 172.21.15.22:0/2185637408 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:36.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15f7fff700 1 --2- 172.21.15.22:0/2185637408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15f806d040 0x7f15f8154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15fca54700 1 -- 172.21.15.22:0/2185637408 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15f80ccd70 msgr2=0x7f15f80c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15fca54700 1 --2- 172.21.15.22:0/2185637408 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15f80ccd70 0x7f15f80c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15fca54700 1 -- 172.21.15.22:0/2185637408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15f806d040 msgr2=0x7f15f8154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15fca54700 1 --2- 172.21.15.22:0/2185637408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15f806d040 0x7f15f8154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.126 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15fca54700 1 -- 172.21.15.22:0/2185637408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f0009b30 con 0x7f15f8159360 2023-07-06T01:21:36.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15fca54700 1 --2- 172.21.15.22:0/2185637408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f8159360 0x7f15f80b4d60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f15f8071d00 tx=0x7f15f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:36.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.121+0000 7f15f4ff9700 1 -- 172.21.15.22:0/2185637408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0004930 con 0x7f15f8159360 2023-07-06T01:21:36.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.122+0000 7f15f4ff9700 1 -- 172.21.15.22:0/2185637408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f00092e0 con 0x7f15f8159360 2023-07-06T01:21:36.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.122+0000 7f15f4ff9700 1 -- 172.21.15.22:0/2185637408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f001a6e0 con 0x7f15f8159360 2023-07-06T01:21:36.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.122+0000 7f15fe4b7700 1 -- 172.21.15.22:0/2185637408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f80c8ca0 con 0x7f15f8159360 2023-07-06T01:21:36.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.122+0000 7f15fe4b7700 1 -- 172.21.15.22:0/2185637408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f80c9260 con 0x7f15f8159360 2023-07-06T01:21:36.128 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.126+0000 7f15f4ff9700 1 -- 172.21.15.22:0/2185637408 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f15f001a840 con 0x7f15f8159360 2023-07-06T01:21:36.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.127+0000 7f15f4ff9700 1 --2- 172.21.15.22:0/2185637408 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f15d806ace0 0x7f15d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.127+0000 7f15f4ff9700 1 -- 172.21.15.22:0/2185637408 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f15f008c970 con 0x7f15f8159360 2023-07-06T01:21:36.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.127+0000 7f15f7fff700 1 --2- 172.21.15.22:0/2185637408 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f15d806ace0 0x7f15d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.127+0000 7f160a63cb80 1 Processor -- start 2023-07-06T01:21:36.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.128+0000 7f160a63cb80 1 -- start start 2023-07-06T01:21:36.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.128+0000 7f15f7fff700 1 --2- 172.21.15.22:0/2185637408 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f15d806ace0 0x7f15d806d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007fd0 tx=0x7f15e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:36.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f160a63cb80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d92824c0 0x5649d92a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f160a63cb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9258880 0x5649d9285240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f160a63cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9282d20 0x5649d9285780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f160a63cb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d915fcb0 con 0x5649d9258880 2023-07-06T01:21:36.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f160a63cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d91b2a40 con 0x5649d9282d20 2023-07-06T01:21:36.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f160a63cb80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649d91abed0 con 0x5649d92824c0 2023-07-06T01:21:36.132 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d92824c0 0x5649d92a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f15f77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9258880 0x5649d9285240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d92824c0 0x5649d92a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:58878/0 (socket says 172.21.15.22:58878) 2023-07-06T01:21:36.133 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f15f7fff700 1 -- 172.21.15.22:0/1681718705 learned_addr learned my addr 172.21.15.22:0/1681718705 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:36.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f15fca54700 1 --2- 172.21.15.22:0/1681718705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9282d20 0x5649d9285780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f15f77fe700 1 -- 172.21.15.22:0/1681718705 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d92824c0 msgr2=0x5649d92a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:36.134 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f15f77fe700 1 --2- 172.21.15.22:0/1681718705 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d92824c0 0x5649d92a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f15f77fe700 1 -- 172.21.15.22:0/1681718705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9282d20 msgr2=0x5649d9285780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:36.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f15f77fe700 1 --2- 172.21.15.22:0/1681718705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9282d20 0x5649d9285780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f15f77fe700 1 -- 172.21.15.22:0/1681718705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d8fd9de0 con 0x5649d9258880 2023-07-06T01:21:36.135 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f15fca54700 1 --2- 172.21.15.22:0/1681718705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9282d20 0x5649d9285780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.129+0000 7f15f77fe700 1 --2- 172.21.15.22:0/1681718705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9258880 0x5649d9285240 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f15f80cd1b0 tx=0x7f15e8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9060be51bd5d17b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.130+0000 7f15edffb700 1 -- 172.21.15.22:0/1681718705 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e800c070 con 0x5649d9258880 2023-07-06T01:21:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.130+0000 7f15edffb700 1 -- 172.21.15.22:0/1681718705 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e8013610 con 0x5649d9258880 2023-07-06T01:21:36.136 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.130+0000 7f15edffb700 1 -- 172.21.15.22:0/1681718705 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e80139c0 con 0x5649d9258880 2023-07-06T01:21:36.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.131+0000 7f160a63cb80 1 -- 172.21.15.22:0/1681718705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9258880 msgr2=0x5649d9285240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.131+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1681718705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9258880 0x5649d9285240 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f15f80cd1b0 tx=0x7f15e8009c00 comp rx=0 tx=0).stop 2023-07-06T01:21:36.142 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.131+0000 7f160a63cb80 1 -- 172.21.15.22:0/1681718705 shutdown_connections 2023-07-06T01:21:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.131+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1681718705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9282d20 0x5649d9285780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.131+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1681718705 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d92824c0 0x5649d92a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.131+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1681718705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9258880 0x5649d9285240 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.131+0000 7f160a63cb80 1 -- 172.21.15.22:0/1681718705 >> 172.21.15.22:0/1681718705 conn(0x5649d924fed0 msgr2=0x5649d92a0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.131+0000 7f160a63cb80 1 -- 172.21.15.22:0/1681718705 shutdown_connections 2023-07-06T01:21:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.131+0000 7f160a63cb80 1 -- 172.21.15.22:0/1681718705 wait complete. 2023-07-06T01:21:36.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.132+0000 7f160a63cb80 1 -- start start 2023-07-06T01:21:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.132+0000 7f160a63cb80 1 Processor -- start 2023-07-06T01:21:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f160a63cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9258880 0x5649d9280a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f160a63cb80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d92824c0 0x5649d9259020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f160a63cb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9282d20 0x5649d9259560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f160a63cb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d914bac0 con 0x5649d9282d20 2023-07-06T01:21:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f160a63cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d914b7c0 con 0x5649d9258880 2023-07-06T01:21:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f160a63cb80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649d902f920 con 0x5649d92824c0 2023-07-06T01:21:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f160a63cb80 10 client.0 _ll_register_callbacks cb 0x5649d9149760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T01:21:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f15f77fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d92824c0 0x5649d9259020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9258880 0x5649d9280a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f15f77fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d92824c0 0x5649d9259020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:58886/0 (socket says 172.21.15.22:58886) 2023-07-06T01:21:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9258880 0x5649d9280a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:36470/0 (socket says 172.21.15.22:36470) 2023-07-06T01:21:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f15f77fe700 1 -- 172.21.15.22:0/1430488401 learned_addr learned my addr 172.21.15.22:0/1430488401 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f15fca54700 1 --2- 172.21.15.22:0/1430488401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9282d20 0x5649d9259560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f15f77fe700 1 -- 172.21.15.22:0/1430488401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9258880 msgr2=0x5649d9280a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f15f77fe700 1 --2- 172.21.15.22:0/1430488401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9258880 0x5649d9280a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f15f77fe700 1 -- 172.21.15.22:0/1430488401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9282d20 msgr2=0x5649d9259560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f15f77fe700 1 --2- 172.21.15.22:0/1430488401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9282d20 0x5649d9259560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.134+0000 7f15f77fe700 1 -- 172.21.15.22:0/1430488401 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f80c9260 con 0x5649d92824c0 2023-07-06T01:21:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.135+0000 7f15f77fe700 1 --2- 172.21.15.22:0/1430488401 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d92824c0 0x5649d9259020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f15e8000c00 tx=0x7f15e8006ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f43f959b23f31ae server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.135+0000 7f15d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.103:3301/0 2023-07-06T01:21:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.135+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1430488401 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8013ea0 con 0x5649d92824c0 2023-07-06T01:21:36.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.135+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1430488401 <== mon.2 v2:172.21.15.103:3301/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f15e8013410 con 0x5649d92824c0 2023-07-06T01:21:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.135+0000 7f160a63cb80 1 -- 172.21.15.22:0/1430488401 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649d924d4e0 con 0x5649d92824c0 2023-07-06T01:21:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.135+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1430488401 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e8006ab0 con 0x5649d92824c0 2023-07-06T01:21:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.135+0000 7f160a63cb80 10 client.4940 fetch_fsmap learned FSMap version 23 2023-07-06T01:21:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.135+0000 7f160a63cb80 1 -- 172.21.15.22:0/1430488401 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x7f15e8009580 con 0x5649d92824c0 2023-07-06T01:21:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.136+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1430488401 <== mon.2 v2:172.21.15.103:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5649d924d4e0 con 0x5649d92824c0 2023-07-06T01:21:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.136+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1430488401 <== mon.2 v2:172.21.15.103:3301/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f15e801c630 con 0x5649d92824c0 2023-07-06T01:21:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.136+0000 7f160a63cb80 10 client.4940 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T01:21:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.136+0000 7f160a63cb80 10 client.4940 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-07-06T01:21:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.136+0000 7f160a63cb80 20 client.4940 populate_metadata read hostname 'smithi022' 2023-07-06T01:21:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.136+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1430488401 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5649d9280fa0 0x5649d9298230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.136+0000 7f160a63cb80 4 client.4940 mds_command: new command op to 5115 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T01:21:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.136+0000 7f160a63cb80 1 -- 172.21.15.22:0/1430488401 --> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649d91a4eb0 con 0x5649d9280fa0 2023-07-06T01:21:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.136+0000 7f15f7fff700 1 --2- 172.21.15.22:0/1430488401 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5649d9280fa0 0x5649d9298230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.136+0000 7f15d6ffd700 10 client.4940 ms_handle_connect on v2:172.21.15.22:6833/612827809 2023-07-06T01:21:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.136+0000 7f15f7fff700 1 --2- 172.21.15.22:0/1430488401 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5649d9280fa0 0x5649d9298230 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a03eb0d7e6a0165a server_cookie=63293a20c2ab1cda in_seq=0 out_seq=0 2023-07-06T01:21:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.137+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1430488401 <== mds.0 v2:172.21.15.22:6833/612827809 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5649d91a4eb0 con 0x5649d9280fa0 2023-07-06T01:21:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.137+0000 7f15d6ffd700 10 client.4940 handle_command_reply: tid=0 2023-07-06T01:21:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.137+0000 7f160a63cb80 1 client.4940 shutdown 2023-07-06T01:21:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.138+0000 7f160a63cb80 1 -- 172.21.15.22:0/1430488401 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d92824c0 msgr2=0x5649d9259020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.138+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1430488401 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d92824c0 0x5649d9259020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f15e8000c00 tx=0x7f15e8006ea0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.138+0000 7f160a63cb80 1 -- 172.21.15.22:0/1430488401 shutdown_connections 2023-07-06T01:21:36.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.138+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1430488401 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5649d9280fa0 0x5649d9298230 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.138+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1430488401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9282d20 0x5649d9259560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.138+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1430488401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9258880 0x5649d9280a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.138+0000 7f15d6ffd700 0 client.4940 ms_handle_reset on v2:172.21.15.22:6833/612827809 2023-07-06T01:21:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.138+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1430488401 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d92824c0 0x5649d9259020 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.138+0000 7f160a63cb80 1 -- 172.21.15.22:0/1430488401 >> 172.21.15.22:0/1430488401 conn(0x5649d924fed0 msgr2=0x5649d9254160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.138+0000 7f160a63cb80 1 -- 172.21.15.22:0/1430488401 shutdown_connections 2023-07-06T01:21:36.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.138+0000 7f160a63cb80 1 -- 172.21.15.22:0/1430488401 wait complete. 2023-07-06T01:21:36.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.138+0000 7f160a63cb80 20 client.4940 trim_cache size 0 max 16384 2023-07-06T01:21:36.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.172+0000 7f160a63cb80 1 Processor -- start 2023-07-06T01:21:36.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.172+0000 7f160a63cb80 1 -- start start 2023-07-06T01:21:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.173+0000 7f160a63cb80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d9258880 0x5649d925c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.173+0000 7f160a63cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d92824c0 0x5649d92ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.173+0000 7f160a63cb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9282d20 0x5649d92890f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.173+0000 7f160a63cb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d92b0170 con 0x5649d9282d20 2023-07-06T01:21:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.173+0000 7f160a63cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d9280800 con 0x5649d92824c0 2023-07-06T01:21:36.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.173+0000 7f160a63cb80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649d9258cc0 con 0x5649d9258880 2023-07-06T01:21:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.173+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d9258880 0x5649d925c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.173+0000 7f15f77fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d92824c0 0x5649d92ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.173+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d9258880 0x5649d925c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:58892/0 (socket says 172.21.15.22:58892) 2023-07-06T01:21:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.173+0000 7f15f7fff700 1 -- 172.21.15.22:0/1337135519 learned_addr learned my addr 172.21.15.22:0/1337135519 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.173+0000 7f15f77fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d92824c0 0x5649d92ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:36480/0 (socket says 172.21.15.22:36480) 2023-07-06T01:21:36.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.173+0000 7f15fca54700 1 --2- 172.21.15.22:0/1337135519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9282d20 0x5649d92890f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.174+0000 7f15f77fe700 1 -- 172.21.15.22:0/1337135519 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d9258880 msgr2=0x5649d925c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.174+0000 7f15f77fe700 1 --2- 172.21.15.22:0/1337135519 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d9258880 0x5649d925c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.174+0000 7f15f77fe700 1 -- 172.21.15.22:0/1337135519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9282d20 msgr2=0x5649d92890f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.174+0000 7f15f77fe700 1 --2- 172.21.15.22:0/1337135519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9282d20 0x5649d92890f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.174+0000 7f15f77fe700 1 -- 172.21.15.22:0/1337135519 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d8fc2c60 con 0x5649d92824c0 2023-07-06T01:21:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.174+0000 7f15f77fe700 1 --2- 172.21.15.22:0/1337135519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d92824c0 0x5649d92ca8b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009870 tx=0x7f15e801f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e5222296b3ca18 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:36.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.174+0000 7f15ee7fc700 1 -- 172.21.15.22:0/1337135519 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e801caf0 con 0x5649d92824c0 2023-07-06T01:21:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.174+0000 7f15ee7fc700 1 -- 172.21.15.22:0/1337135519 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e801c630 con 0x5649d92824c0 2023-07-06T01:21:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.174+0000 7f15ee7fc700 1 -- 172.21.15.22:0/1337135519 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8025650 con 0x5649d92824c0 2023-07-06T01:21:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.175+0000 7f160a63cb80 1 -- 172.21.15.22:0/1337135519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d92824c0 msgr2=0x5649d92ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.175+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1337135519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d92824c0 0x5649d92ca8b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009870 tx=0x7f15e801f040 comp rx=0 tx=0).stop 2023-07-06T01:21:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.175+0000 7f160a63cb80 1 -- 172.21.15.22:0/1337135519 shutdown_connections 2023-07-06T01:21:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.175+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1337135519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d9282d20 0x5649d92890f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.175+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1337135519 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d9258880 0x5649d925c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.175+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1337135519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d92824c0 0x5649d92ca8b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.175+0000 7f160a63cb80 1 -- 172.21.15.22:0/1337135519 >> 172.21.15.22:0/1337135519 conn(0x5649d924fed0 msgr2=0x5649d9257310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.175+0000 7f160a63cb80 1 -- 172.21.15.22:0/1337135519 shutdown_connections 2023-07-06T01:21:36.180 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.175+0000 7f160a63cb80 1 -- 172.21.15.22:0/1337135519 wait complete. 2023-07-06T01:21:36.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.176+0000 7f160a63cb80 1 -- start start 2023-07-06T01:21:36.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.177+0000 7f160a63cb80 1 Processor -- start 2023-07-06T01:21:36.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.177+0000 7f160a63cb80 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d9258880 0x5649d92f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.177+0000 7f160a63cb80 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d92824c0 0x5649d92a3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.177+0000 7f160a63cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9282d20 0x5649d9258e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.177+0000 7f160a63cb80 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d927df80 con 0x5649d92824c0 2023-07-06T01:21:36.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.177+0000 7f160a63cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d92f5f40 con 0x5649d9282d20 2023-07-06T01:21:36.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.177+0000 7f160a63cb80 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649d92a3ec0 con 0x5649d9258880 2023-07-06T01:21:36.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f160a63cb80 10 client.0 _ll_register_callbacks cb 0x7f15f0003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T01:21:36.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15f77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d92824c0 0x5649d92a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15f77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d92824c0 0x5649d92a3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47762/0 (socket says 172.21.15.22:47762) 2023-07-06T01:21:36.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15f7fff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d9258880 0x5649d92f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15f77fe700 1 -- 172.21.15.22:0/1925890518 learned_addr learned my addr 172.21.15.22:0/1925890518 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:36.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15fca54700 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9282d20 0x5649d9258e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15f77fe700 1 -- 172.21.15.22:0/1925890518 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d9258880 msgr2=0x5649d92f53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:36.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15f77fe700 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d9258880 0x5649d92f53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.190 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15f77fe700 1 -- 172.21.15.22:0/1925890518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9282d20 msgr2=0x5649d9258e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:36.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15f77fe700 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9282d20 0x5649d9258e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15f77fe700 1 -- 172.21.15.22:0/1925890518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e8009580 con 0x5649d92824c0 2023-07-06T01:21:36.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15fca54700 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9282d20 0x5649d9258e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:36.191 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15f7fff700 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d9258880 0x5649d92f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:36.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15f77fe700 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d92824c0 0x5649d92a3420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f15e8027040 tx=0x7f15e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8388c07b800928aa server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:36.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2023-07-06T01:21:36.192 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1925890518 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8025e10 con 0x5649d92824c0 2023-07-06T01:21:36.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1925890518 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f15e8000e40 con 0x5649d92824c0 2023-07-06T01:21:36.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.178+0000 7f160a63cb80 1 -- 172.21.15.22:0/1925890518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649d924d3f0 con 0x5649d92824c0 2023-07-06T01:21:36.193 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.179+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1925890518 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e801c7a0 con 0x5649d92824c0 2023-07-06T01:21:36.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.179+0000 7f160a63cb80 10 client.5385 fetch_fsmap learned FSMap version 23 2023-07-06T01:21:36.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.179+0000 7f160a63cb80 1 -- 172.21.15.22:0/1925890518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5649d9000250 con 0x5649d92824c0 2023-07-06T01:21:36.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.179+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1925890518 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5649d924d3f0 con 0x5649d92824c0 2023-07-06T01:21:36.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.179+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1925890518 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f15e801ca00 con 0x5649d92824c0 2023-07-06T01:21:36.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.179+0000 7f160a63cb80 10 client.5385 fetch_fsmap finished waiting for FSMap version 23 2023-07-06T01:21:36.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.179+0000 7f160a63cb80 10 client.5385 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-07-06T01:21:36.194 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.179+0000 7f160a63cb80 20 client.5385 populate_metadata read hostname 'smithi022' 2023-07-06T01:21:36.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.179+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5649d92a3960 0x5649d9259370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.179+0000 7f160a63cb80 4 client.5385 mds_command: new command op to 5115 tid=0[{"prefix": "scrub status"}] 2023-07-06T01:21:36.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.179+0000 7f160a63cb80 1 -- 172.21.15.22:0/1925890518 --> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f15e8013610 con 0x5649d92a3960 2023-07-06T01:21:36.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.179+0000 7f15f7fff700 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5649d92a3960 0x5649d9259370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.180+0000 7f15d6ffd700 10 client.5385 ms_handle_connect on v2:172.21.15.22:6833/612827809 2023-07-06T01:21:36.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.180+0000 7f15f7fff700 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5649d92a3960 0x5649d9259370 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520c9fd52d718d54 server_cookie=3af4021ce79e3327 in_seq=0 out_seq=0 2023-07-06T01:21:36.195 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.180+0000 7f15d6ffd700 1 -- 172.21.15.22:0/1925890518 <== mds.0 v2:172.21.15.22:6833/612827809 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+77 (crc 0 0 0) 0x7f15e8013610 con 0x5649d92a3960 2023-07-06T01:21:36.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.180+0000 7f15d6ffd700 10 client.5385 handle_command_reply: tid=0 2023-07-06T01:21:36.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.180+0000 7f160a63cb80 1 client.5385 shutdown 2023-07-06T01:21:36.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.181+0000 7f160a63cb80 1 -- 172.21.15.22:0/1925890518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d92824c0 msgr2=0x5649d92a3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.181+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d92824c0 0x5649d92a3420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f15e8027040 tx=0x7f15e8008040 comp rx=0 tx=0).stop 2023-07-06T01:21:36.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.181+0000 7f160a63cb80 1 -- 172.21.15.22:0/1925890518 shutdown_connections 2023-07-06T01:21:36.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.181+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5649d9282d20 0x5649d9258e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.196 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.181+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x5649d9258880 0x5649d92f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.181+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.22:6833/612827809,v1:172.21.15.22:6835/612827809] conn(0x5649d92a3960 0x5649d9259370 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.181+0000 7f160a63cb80 1 --2- 172.21.15.22:0/1925890518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5649d92824c0 0x5649d92a3420 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.181+0000 7f160a63cb80 1 -- 172.21.15.22:0/1925890518 >> 172.21.15.22:0/1925890518 conn(0x5649d924fed0 msgr2=0x5649d92ac7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:36.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.181+0000 7f15d6ffd700 0 client.5385 ms_handle_reset on v2:172.21.15.22:6833/612827809 2023-07-06T01:21:36.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.181+0000 7f160a63cb80 1 -- 172.21.15.22:0/1925890518 shutdown_connections 2023-07-06T01:21:36.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.181+0000 7f160a63cb80 1 -- 172.21.15.22:0/1925890518 wait complete. 2023-07-06T01:21:36.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.181+0000 7f160a63cb80 20 client.5385 trim_cache size 0 max 16384 2023-07-06T01:21:36.198 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:21:36.198 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "scrub active (939 inodes in the stack)", 2023-07-06T01:21:36.198 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {} 2023-07-06T01:21:36.198 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:21:36.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.182+0000 7f15fe4b7700 1 -- 172.21.15.22:0/2185637408 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f15d806ace0 msgr2=0x7f15d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.182+0000 7f15fe4b7700 1 --2- 172.21.15.22:0/2185637408 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f15d806ace0 0x7f15d806d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007fd0 tx=0x7f15e0009040 comp rx=0 tx=0).stop 2023-07-06T01:21:36.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.182+0000 7f15fe4b7700 1 -- 172.21.15.22:0/2185637408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f8159360 msgr2=0x7f15f80b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.182+0000 7f15fe4b7700 1 --2- 172.21.15.22:0/2185637408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f8159360 0x7f15f80b4d60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f15f8071d00 tx=0x7f15f0004ad0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.182+0000 7f15fe4b7700 1 -- 172.21.15.22:0/2185637408 shutdown_connections 2023-07-06T01:21:36.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.182+0000 7f15fe4b7700 1 --2- 172.21.15.22:0/2185637408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15f8159360 0x7f15f80b4d60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.182+0000 7f15fe4b7700 1 --2- 172.21.15.22:0/2185637408 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f15d806ace0 0x7f15d806d190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.182+0000 7f15fe4b7700 1 --2- 172.21.15.22:0/2185637408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15f806d040 0x7f15f8154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.182+0000 7f15fe4b7700 1 --2- 172.21.15.22:0/2185637408 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15f80ccd70 0x7f15f80c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.182+0000 7f15fe4b7700 1 -- 172.21.15.22:0/2185637408 >> 172.21.15.22:0/2185637408 conn(0x7f15f80b1890 msgr2=0x7f15f806d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:36.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.182+0000 7f15fe4b7700 1 -- 172.21.15.22:0/2185637408 shutdown_connections 2023-07-06T01:21:36.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.182+0000 7f15fe4b7700 1 -- 172.21.15.22:0/2185637408 wait complete. 2023-07-06T01:21:36.201 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T01:21:36.202 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T01:21:36.255 DEBUG:teuthology.orchestra.run.smithi103:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T01:21:36.298 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T01:21:36.299 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:21:36.299 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:21:36.319 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:21:36.319 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:21:36.394 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:21:36.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.754+0000 7f4c85f7c700 1 Processor -- start 2023-07-06T01:21:36.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.754+0000 7f4c85f7c700 1 -- start start 2023-07-06T01:21:36.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.755+0000 7f4c85f7c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c8006d3a0 0x7f4c800cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.755+0000 7f4c85f7c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4c8006b9e0 0x7f4c8006be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.757 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.755+0000 7f4c85f7c700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c800cb1c0 con 0x7f4c800c91d0 2023-07-06T01:21:36.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.755+0000 7f4c85f7c700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c800cb410 con 0x7f4c8006b9e0 2023-07-06T01:21:36.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.755+0000 7f4c85f7c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8015d4e0 con 0x7f4c8006d3a0 2023-07-06T01:21:36.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.755+0000 7f4c7ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4c8006b9e0 0x7f4c8006be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.755+0000 7f4c7ffff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f4c8006b9e0 0x7f4c8006be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47770/0 (socket says 172.21.15.22:47770) 2023-07-06T01:21:36.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.755+0000 7f4c7ffff700 1 -- 172.21.15.22:0/1181547456 learned_addr learned my addr 172.21.15.22:0/1181547456 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:36.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.756+0000 7f4c7f7fe700 1 --2- 172.21.15.22:0/1181547456 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c8006d3a0 0x7f4c800cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.757+0000 7f4c7f7fe700 1 -- 172.21.15.22:0/1181547456 >> v2:172.21.15.22:3300/0 conn(0x7f4c8006b9e0 msgr2=0x7f4c8006be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:36.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.757+0000 7f4c7f7fe700 1 --2- 172.21.15.22:0/1181547456 >> v2:172.21.15.22:3300/0 conn(0x7f4c8006b9e0 0x7f4c8006be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.757+0000 7f4c7f7fe700 1 -- 172.21.15.22:0/1181547456 >> v1:172.21.15.103:6789/0 conn(0x7f4c800c91d0 legacy=0x7f4c8006d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.757+0000 7f4c7f7fe700 1 -- 172.21.15.22:0/1181547456 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c800cdd50 con 0x7f4c8006d3a0 2023-07-06T01:21:36.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.757+0000 7f4c7f7fe700 1 --2- 172.21.15.22:0/1181547456 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c8006d3a0 0x7f4c800cac80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009840 tx=0x7f4c74009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfc994f2b7808b39 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:36.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.757+0000 7f4c7e7fc700 1 -- 172.21.15.22:0/1181547456 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400f880 con 0x7f4c8006d3a0 2023-07-06T01:21:36.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.757+0000 7f4c7e7fc700 1 -- 172.21.15.22:0/1181547456 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7400f9e0 con 0x7f4c8006d3a0 2023-07-06T01:21:36.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.757+0000 7f4c7e7fc700 1 -- 172.21.15.22:0/1181547456 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7400fd70 con 0x7f4c8006d3a0 2023-07-06T01:21:36.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.758+0000 7f4c85f7c700 1 -- 172.21.15.22:0/1181547456 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c8006d3a0 msgr2=0x7f4c800cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.758+0000 7f4c85f7c700 1 --2- 172.21.15.22:0/1181547456 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c8006d3a0 0x7f4c800cac80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009840 tx=0x7f4c74009b50 comp rx=0 tx=0).stop 2023-07-06T01:21:36.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.759+0000 7f4c85f7c700 1 -- 172.21.15.22:0/1181547456 shutdown_connections 2023-07-06T01:21:36.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.759+0000 7f4c85f7c700 1 --2- 172.21.15.22:0/1181547456 >> v2:172.21.15.22:3300/0 conn(0x7f4c8006b9e0 0x7f4c8006be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.759+0000 7f4c85f7c700 1 --2- 172.21.15.22:0/1181547456 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c8006d3a0 0x7f4c800cac80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.759+0000 7f4c85f7c700 1 -- 172.21.15.22:0/1181547456 >> 172.21.15.22:0/1181547456 conn(0x7f4c800b13a0 msgr2=0x7f4c800ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:36.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.760+0000 7f4c85f7c700 1 -- 172.21.15.22:0/1181547456 shutdown_connections 2023-07-06T01:21:36.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.760+0000 7f4c85f7c700 1 -- 172.21.15.22:0/1181547456 wait complete. 2023-07-06T01:21:36.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.762+0000 7f4c85f7c700 1 Processor -- start 2023-07-06T01:21:36.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.762+0000 7f4c85f7c700 1 -- start start 2023-07-06T01:21:36.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.763+0000 7f4c85f7c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c8006b9e0 0x7f4c8014de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.763+0000 7f4c85f7c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c8006d3a0 0x7f4c800c3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.763+0000 7f4c85f7c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c800c91d0 0x7f4c800c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.763+0000 7f4c85f7c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c800c7c40 con 0x7f4c800c91d0 2023-07-06T01:21:36.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.763+0000 7f4c85f7c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c800c6940 con 0x7f4c8006d3a0 2023-07-06T01:21:36.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.763+0000 7f4c85f7c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c800c75f0 con 0x7f4c8006b9e0 2023-07-06T01:21:36.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.764+0000 7f4c7ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c800c91d0 0x7f4c800c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.764+0000 7f4c7ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c800c91d0 0x7f4c800c38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47778/0 (socket says 172.21.15.22:47778) 2023-07-06T01:21:36.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.764+0000 7f4c7ffff700 1 -- 172.21.15.22:0/3534693485 learned_addr learned my addr 172.21.15.22:0/3534693485 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:36.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.764+0000 7f4c7f7fe700 1 --2- 172.21.15.22:0/3534693485 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c8006b9e0 0x7f4c8014de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.764+0000 7f4c7ffff700 1 -- 172.21.15.22:0/3534693485 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c8006b9e0 msgr2=0x7f4c8014de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.764+0000 7f4c7ffff700 1 --2- 172.21.15.22:0/3534693485 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c8006b9e0 0x7f4c8014de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.764+0000 7f4c7ffff700 1 -- 172.21.15.22:0/3534693485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c8006d3a0 msgr2=0x7f4c800c3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:36.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.764+0000 7f4c7ffff700 1 --2- 172.21.15.22:0/3534693485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c8006d3a0 0x7f4c800c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:36.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.764+0000 7f4c7ffff700 1 -- 172.21.15.22:0/3534693485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c740094a0 con 0x7f4c800c91d0 2023-07-06T01:21:36.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.765+0000 7f4c7ffff700 1 --2- 172.21.15.22:0/3534693485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c800c91d0 0x7f4c800c38b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4c780042d0 tx=0x7f4c780043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:36.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.765+0000 7f4c67fff700 1 -- 172.21.15.22:0/3534693485 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78003ad0 con 0x7f4c800c91d0 2023-07-06T01:21:36.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.765+0000 7f4c67fff700 1 -- 172.21.15.22:0/3534693485 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7800c730 con 0x7f4c800c91d0 2023-07-06T01:21:36.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.765+0000 7f4c67fff700 1 -- 172.21.15.22:0/3534693485 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7800cae0 con 0x7f4c800c91d0 2023-07-06T01:21:36.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.766+0000 7f4c85f7c700 1 -- 172.21.15.22:0/3534693485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c800c78d0 con 0x7f4c800c91d0 2023-07-06T01:21:36.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.766+0000 7f4c85f7c700 1 -- 172.21.15.22:0/3534693485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8014e570 con 0x7f4c800c91d0 2023-07-06T01:21:36.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.767+0000 7f4c85f7c700 1 -- 172.21.15.22:0/3534693485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c800c7430 con 0x7f4c800c91d0 2023-07-06T01:21:36.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.769+0000 7f4c67fff700 1 -- 172.21.15.22:0/3534693485 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c78016070 con 0x7f4c800c91d0 2023-07-06T01:21:36.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.769+0000 7f4c67fff700 1 --2- 172.21.15.22:0/3534693485 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4c6c06ad30 0x7f4c6c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:36.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.769+0000 7f4c67fff700 1 -- 172.21.15.22:0/3534693485 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6147+0+0 (secure 0 0 0) 0x7f4c78089e00 con 0x7f4c800c91d0 2023-07-06T01:21:36.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.777+0000 7f4c7f7fe700 1 --2- 172.21.15.22:0/3534693485 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4c6c06ad30 0x7f4c6c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:36.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.777+0000 7f4c67fff700 1 -- 172.21.15.22:0/3534693485 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4c78054e30 con 0x7f4c800c91d0 2023-07-06T01:21:36.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:36.779+0000 7f4c7f7fe700 1 --2- 172.21.15.22:0/3534693485 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4c6c06ad30 0x7f4c6c06d1e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009b90 tx=0x7f4c74008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:37.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.088+0000 7f4c85f7c700 1 -- 172.21.15.22:0/3534693485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c800744b0 con 0x7f4c800c91d0 2023-07-06T01:21:37.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.089+0000 7f4c67fff700 1 -- 172.21.15.22:0/3534693485 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18382 (secure 0 0 0) 0x7f4c78058450 con 0x7f4c800c91d0 2023-07-06T01:21:37.091 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:21:37.091 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":37,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:21:36.323438+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:20:22.043370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T01:20:23.225298+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:21:37.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.091+0000 7f4c85f7c700 1 -- 172.21.15.22:0/3534693485 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4c6c06ad30 msgr2=0x7f4c6c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.091+0000 7f4c85f7c700 1 --2- 172.21.15.22:0/3534693485 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4c6c06ad30 0x7f4c6c06d1e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009b90 tx=0x7f4c74008030 comp rx=0 tx=0).stop 2023-07-06T01:21:37.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.091+0000 7f4c85f7c700 1 -- 172.21.15.22:0/3534693485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c800c91d0 msgr2=0x7f4c800c38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.091+0000 7f4c85f7c700 1 --2- 172.21.15.22:0/3534693485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c800c91d0 0x7f4c800c38b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4c780042d0 tx=0x7f4c780043b0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.091+0000 7f4c85f7c700 1 -- 172.21.15.22:0/3534693485 shutdown_connections 2023-07-06T01:21:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.091+0000 7f4c85f7c700 1 --2- 172.21.15.22:0/3534693485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c800c91d0 0x7f4c800c38b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.091+0000 7f4c85f7c700 1 --2- 172.21.15.22:0/3534693485 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c8006b9e0 0x7f4c8014de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.091+0000 7f4c85f7c700 1 --2- 172.21.15.22:0/3534693485 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4c6c06ad30 0x7f4c6c06d1e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.091+0000 7f4c85f7c700 1 --2- 172.21.15.22:0/3534693485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c8006d3a0 0x7f4c800c3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.091+0000 7f4c85f7c700 1 -- 172.21.15.22:0/3534693485 >> 172.21.15.22:0/3534693485 conn(0x7f4c800b13a0 msgr2=0x7f4c800aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.091+0000 7f4c85f7c700 1 -- 172.21.15.22:0/3534693485 shutdown_connections 2023-07-06T01:21:37.094 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.091+0000 7f4c85f7c700 1 -- 172.21.15.22:0/3534693485 wait complete. 2023-07-06T01:21:37.105 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T01:20:22.043370+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T01:20:23.225298+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:21:37.106 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:21:37.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.233+0000 7fb0743bd700 1 Processor -- start 2023-07-06T01:21:37.235 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.233+0000 7fb0743bd700 1 -- start start 2023-07-06T01:21:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.234+0000 7fb0743bd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb06c15a710 0x7fb06c0ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.234+0000 7fb0743bd700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb06c0af270 con 0x7fb06c0aecb0 2023-07-06T01:21:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.234+0000 7fb0743bd700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb06c0af550 con 0x7fb06c0cef70 2023-07-06T01:21:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.234+0000 7fb0743bd700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06c0af830 con 0x7fb06c15a710 2023-07-06T01:21:37.236 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.234+0000 7fb072159700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb06c15a710 0x7fb06c0ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.234+0000 7fb072159700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb06c15a710 0x7fb06c0ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47782/0 (socket says 172.21.15.22:47782) 2023-07-06T01:21:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.234+0000 7fb071958700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb06c0aecb0 0x7fb06c0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38526/0 (socket says 172.21.15.22:38526) 2023-07-06T01:21:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.234+0000 7fb072159700 1 -- 172.21.15.22:0/3907425232 learned_addr learned my addr 172.21.15.22:0/3907425232 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.234+0000 7fb072159700 1 -- 172.21.15.22:0/3907425232 >> v1:172.21.15.103:6789/0 conn(0x7fb06c0cef70 legacy=0x7fb06c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.234+0000 7fb072159700 1 -- 172.21.15.22:0/3907425232 >> v1:172.21.15.22:6789/0 conn(0x7fb06c0aecb0 legacy=0x7fb06c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.234+0000 7fb072159700 1 -- 172.21.15.22:0/3907425232 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c0cb2e0 con 0x7fb06c15a710 2023-07-06T01:21:37.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.234+0000 7fb072159700 1 --2- 172.21.15.22:0/3907425232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06c15a710 0x7fb06c0ae770 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb0680098f0 tx=0x7fb068009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b83ebd695d19f70 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.234+0000 7fb071157700 1 -- 172.21.15.22:0/3907425232 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06800b8a0 con 0x7fb06c15a710 2023-07-06T01:21:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.235+0000 7fb071157700 1 -- 172.21.15.22:0/3907425232 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06800ba00 con 0x7fb06c15a710 2023-07-06T01:21:37.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.235+0000 7fb071157700 1 -- 172.21.15.22:0/3907425232 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06800bd90 con 0x7fb06c15a710 2023-07-06T01:21:37.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.235+0000 7fb0743bd700 1 -- 172.21.15.22:0/3907425232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06c15a710 msgr2=0x7fb06c0ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.235+0000 7fb0743bd700 1 --2- 172.21.15.22:0/3907425232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06c15a710 0x7fb06c0ae770 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb0680098f0 tx=0x7fb068009c00 comp rx=0 tx=0).stop 2023-07-06T01:21:37.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.235+0000 7fb0743bd700 1 -- 172.21.15.22:0/3907425232 shutdown_connections 2023-07-06T01:21:37.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.235+0000 7fb0743bd700 1 --2- 172.21.15.22:0/3907425232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06c15a710 0x7fb06c0ae770 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.235+0000 7fb0743bd700 1 -- 172.21.15.22:0/3907425232 >> 172.21.15.22:0/3907425232 conn(0x7fb06c0999f0 msgr2=0x7fb06c099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:37.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.235+0000 7fb0743bd700 1 -- 172.21.15.22:0/3907425232 shutdown_connections 2023-07-06T01:21:37.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.235+0000 7fb0743bd700 1 -- 172.21.15.22:0/3907425232 wait complete. 2023-07-06T01:21:37.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.236+0000 7fb0743bd700 1 Processor -- start 2023-07-06T01:21:37.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.236+0000 7fb0743bd700 1 -- start start 2023-07-06T01:21:37.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb0743bd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06c0aecb0 0x7fb06c1f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb0743bd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb06c0cef70 0x7fb06c0b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb0743bd700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb06c15a710 0x7fb06c14fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb0743bd700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c1f1090 con 0x7fb06c0aecb0 2023-07-06T01:21:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb0743bd700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c0aff10 con 0x7fb06c0cef70 2023-07-06T01:21:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb0743bd700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c0b01f0 con 0x7fb06c15a710 2023-07-06T01:21:37.242 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb072159700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06c0aecb0 0x7fb06c1f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb072159700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06c0aecb0 0x7fb06c1f0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47788/0 (socket says 172.21.15.22:47788) 2023-07-06T01:21:37.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb072159700 1 -- 172.21.15.22:0/2863712144 learned_addr learned my addr 172.21.15.22:0/2863712144 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:37.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb071958700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb06c0cef70 0x7fb06c0b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb07295a700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb06c15a710 0x7fb06c14fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb072159700 1 -- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb06c15a710 msgr2=0x7fb06c14fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb072159700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb06c15a710 0x7fb06c14fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb072159700 1 -- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb06c0cef70 msgr2=0x7fb06c0b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.243 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb072159700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb06c0cef70 0x7fb06c0b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.237+0000 7fb072159700 1 -- 172.21.15.22:0/2863712144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb068009550 con 0x7fb06c0aecb0 2023-07-06T01:21:37.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.238+0000 7fb072159700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06c0aecb0 0x7fb06c1f0b50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb068009130 tx=0x7fb068004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:37.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.238+0000 7fb062ffd700 1 -- 172.21.15.22:0/2863712144 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0680036a0 con 0x7fb06c0aecb0 2023-07-06T01:21:37.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.238+0000 7fb062ffd700 1 -- 172.21.15.22:0/2863712144 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06801e070 con 0x7fb06c0aecb0 2023-07-06T01:21:37.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.238+0000 7fb0743bd700 1 -- 172.21.15.22:0/2863712144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06c0b04d0 con 0x7fb06c0aecb0 2023-07-06T01:21:37.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.238+0000 7fb062ffd700 1 -- 172.21.15.22:0/2863712144 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb068003a60 con 0x7fb06c0aecb0 2023-07-06T01:21:37.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.238+0000 7fb0743bd700 1 -- 172.21.15.22:0/2863712144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06c0b0a90 con 0x7fb06c0aecb0 2023-07-06T01:21:37.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.239+0000 7fb0743bd700 1 -- 172.21.15.22:0/2863712144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06c0ada80 con 0x7fb06c0aecb0 2023-07-06T01:21:37.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.239+0000 7fb062ffd700 1 -- 172.21.15.22:0/2863712144 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb068003bc0 con 0x7fb06c0aecb0 2023-07-06T01:21:37.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.240+0000 7fb062ffd700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb05006ace0 0x7fb05006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.240+0000 7fb062ffd700 1 -- 172.21.15.22:0/2863712144 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6147+0+0 (secure 0 0 0) 0x7fb068020070 con 0x7fb06c0aecb0 2023-07-06T01:21:37.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.240+0000 7fb071958700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb05006ace0 0x7fb05006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.240+0000 7fb071958700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb05006ace0 0x7fb05006d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0b6be0 tx=0x7fb05c007d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:37.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.244+0000 7fb062ffd700 1 -- 172.21.15.22:0/2863712144 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb068055b90 con 0x7fb06c0aecb0 2023-07-06T01:21:37.473 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.472+0000 7fb0743bd700 1 -- 172.21.15.22:0/2863712144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb06c0744b0 con 0x7fb06c0aecb0 2023-07-06T01:21:37.474 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.472+0000 7fb062ffd700 1 -- 172.21.15.22:0/2863712144 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb06802a020 con 0x7fb06c0aecb0 2023-07-06T01:21:37.474 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:21:37.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.474+0000 7fb0743bd700 1 -- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb05006ace0 msgr2=0x7fb05006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.476 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.474+0000 7fb0743bd700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb05006ace0 0x7fb05006d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0b6be0 tx=0x7fb05c007d10 comp rx=0 tx=0).stop 2023-07-06T01:21:37.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.474+0000 7fb0743bd700 1 -- 172.21.15.22:0/2863712144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06c0aecb0 msgr2=0x7fb06c1f0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.474+0000 7fb0743bd700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06c0aecb0 0x7fb06c1f0b50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb068009130 tx=0x7fb068004750 comp rx=0 tx=0).stop 2023-07-06T01:21:37.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.474+0000 7fb0743bd700 1 -- 172.21.15.22:0/2863712144 shutdown_connections 2023-07-06T01:21:37.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.474+0000 7fb0743bd700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb06c15a710 0x7fb06c14fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.474+0000 7fb0743bd700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06c0aecb0 0x7fb06c1f0b50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.474+0000 7fb0743bd700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb05006ace0 0x7fb05006d190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.478 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.474+0000 7fb0743bd700 1 --2- 172.21.15.22:0/2863712144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb06c0cef70 0x7fb06c0b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.474+0000 7fb0743bd700 1 -- 172.21.15.22:0/2863712144 >> 172.21.15.22:0/2863712144 conn(0x7fb06c0999f0 msgr2=0x7fb06c0ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:37.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.474+0000 7fb0743bd700 1 -- 172.21.15.22:0/2863712144 shutdown_connections 2023-07-06T01:21:37.479 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.474+0000 7fb0743bd700 1 -- 172.21.15.22:0/2863712144 wait complete. 2023-07-06T01:21:37.488 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:21:37.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.615+0000 7feb5e141700 1 Processor -- start 2023-07-06T01:21:37.617 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.615+0000 7feb5e141700 1 -- start start 2023-07-06T01:21:37.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.616+0000 7feb5e141700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7feb58159360 0x7feb5815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.616+0000 7feb5e141700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb580ccae0 0x7feb58071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.616+0000 7feb5e141700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb580cf230 con 0x7feb580cc660 2023-07-06T01:21:37.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.616+0000 7feb5e141700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5815df20 con 0x7feb58159360 2023-07-06T01:21:37.619 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.616+0000 7feb5e141700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb580ce620 con 0x7feb580ccae0 2023-07-06T01:21:37.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb57fff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb580ccae0 0x7feb58071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb577fe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7feb58159360 0x7feb5815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb577fe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7feb58159360 0x7feb5815d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:36520/0 (socket says 172.21.15.22:36520) 2023-07-06T01:21:37.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb56ffd700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7feb580cc660 0x7feb58070a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:40604/0 (socket says 172.21.15.22:40604) 2023-07-06T01:21:37.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb56ffd700 1 -- 172.21.15.22:0/709901300 learned_addr learned my addr 172.21.15.22:0/709901300 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:37.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb57fff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb580ccae0 0x7feb58071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:58936/0 (socket says 172.21.15.22:58936) 2023-07-06T01:21:37.620 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb567fc700 1 -- 172.21.15.22:0/709901300 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916979956 0 0) 0x7feb580cf230 con 0x7feb580cc660 2023-07-06T01:21:37.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb57fff700 1 -- 172.21.15.22:0/709901300 >> v2:172.21.15.103:3300/0 conn(0x7feb58159360 msgr2=0x7feb5815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:37.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb57fff700 1 --2- 172.21.15.22:0/709901300 >> v2:172.21.15.103:3300/0 conn(0x7feb58159360 0x7feb5815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb57fff700 1 -- 172.21.15.22:0/709901300 >> v1:172.21.15.103:6789/0 conn(0x7feb580cc660 legacy=0x7feb58070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb57fff700 1 -- 172.21.15.22:0/709901300 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb580ce900 con 0x7feb580ccae0 2023-07-06T01:21:37.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb577fe700 1 --2- 172.21.15.22:0/709901300 >> v2:172.21.15.103:3300/0 conn(0x7feb58159360 0x7feb5815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:37.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb57fff700 1 --2- 172.21.15.22:0/709901300 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb580ccae0 0x7feb58071130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feb50009d90 tx=0x7feb5000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8f7432b5ee1d0d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:37.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb567fc700 1 -- 172.21.15.22:0/709901300 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5000e070 con 0x7feb580ccae0 2023-07-06T01:21:37.621 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb567fc700 1 -- 172.21.15.22:0/709901300 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5000bb20 con 0x7feb580ccae0 2023-07-06T01:21:37.622 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.617+0000 7feb567fc700 1 -- 172.21.15.22:0/709901300 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb50012430 con 0x7feb580ccae0 2023-07-06T01:21:37.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.618+0000 7feb5e141700 1 -- 172.21.15.22:0/709901300 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb580ccae0 msgr2=0x7feb58071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.618+0000 7feb5e141700 1 --2- 172.21.15.22:0/709901300 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb580ccae0 0x7feb58071130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feb50009d90 tx=0x7feb5000b330 comp rx=0 tx=0).stop 2023-07-06T01:21:37.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.618+0000 7feb5e141700 1 -- 172.21.15.22:0/709901300 shutdown_connections 2023-07-06T01:21:37.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.618+0000 7feb5e141700 1 --2- 172.21.15.22:0/709901300 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb580ccae0 0x7feb58071130 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.618+0000 7feb5e141700 1 --2- 172.21.15.22:0/709901300 >> v2:172.21.15.103:3300/0 conn(0x7feb58159360 0x7feb5815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.624 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.618+0000 7feb5e141700 1 -- 172.21.15.22:0/709901300 >> 172.21.15.22:0/709901300 conn(0x7feb580b1890 msgr2=0x7feb580b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:37.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.618+0000 7feb5e141700 1 -- 172.21.15.22:0/709901300 shutdown_connections 2023-07-06T01:21:37.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.618+0000 7feb5e141700 1 -- 172.21.15.22:0/709901300 wait complete. 2023-07-06T01:21:37.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.619+0000 7feb5e141700 1 Processor -- start 2023-07-06T01:21:37.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.619+0000 7feb5e141700 1 -- start start 2023-07-06T01:21:37.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.620+0000 7feb5e141700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb580cc660 0x7feb581f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.620+0000 7feb5e141700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb580ccae0 0x7feb581f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.620+0000 7feb5e141700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb58159360 0x7feb5815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.625 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.620+0000 7feb5e141700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb580c6d70 con 0x7feb580ccae0 2023-07-06T01:21:37.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.620+0000 7feb5e141700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb580c6fc0 con 0x7feb580cc660 2023-07-06T01:21:37.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.620+0000 7feb5e141700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb580c72a0 con 0x7feb58159360 2023-07-06T01:21:37.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.620+0000 7feb56ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb580ccae0 0x7feb581f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.620+0000 7feb56ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb580ccae0 0x7feb581f3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47802/0 (socket says 172.21.15.22:47802) 2023-07-06T01:21:37.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.620+0000 7feb56ffd700 1 -- 172.21.15.22:0/1578714734 learned_addr learned my addr 172.21.15.22:0/1578714734 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:37.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.620+0000 7feb577fe700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb580cc660 0x7feb581f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.626 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.620+0000 7feb57fff700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb58159360 0x7feb5815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.620+0000 7feb56ffd700 1 -- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb58159360 msgr2=0x7feb5815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.621+0000 7feb56ffd700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb58159360 0x7feb5815b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.621+0000 7feb56ffd700 1 -- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb580cc660 msgr2=0x7feb581f0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.621+0000 7feb56ffd700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb580cc660 0x7feb581f0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.621+0000 7feb56ffd700 1 -- 172.21.15.22:0/1578714734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c0094a0 con 0x7feb580ccae0 2023-07-06T01:21:37.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.621+0000 7feb57fff700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb58159360 0x7feb5815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:37.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.621+0000 7feb56ffd700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb580ccae0 0x7feb581f3ae0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7feb480011e0 tx=0x7feb4800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:37.627 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.621+0000 7feb3ffff700 1 -- 172.21.15.22:0/1578714734 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48004c80 con 0x7feb580ccae0 2023-07-06T01:21:37.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.621+0000 7feb3ffff700 1 -- 172.21.15.22:0/1578714734 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb48004e00 con 0x7feb580ccae0 2023-07-06T01:21:37.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.621+0000 7feb3ffff700 1 -- 172.21.15.22:0/1578714734 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48007310 con 0x7feb580ccae0 2023-07-06T01:21:37.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.621+0000 7feb5e141700 1 -- 172.21.15.22:0/1578714734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb50009a70 con 0x7feb580ccae0 2023-07-06T01:21:37.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.621+0000 7feb5e141700 1 -- 172.21.15.22:0/1578714734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb580c7860 con 0x7feb580ccae0 2023-07-06T01:21:37.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.622+0000 7feb5e141700 1 -- 172.21.15.22:0/1578714734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb580744b0 con 0x7feb580ccae0 2023-07-06T01:21:37.628 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.622+0000 7feb3ffff700 1 -- 172.21.15.22:0/1578714734 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feb48007470 con 0x7feb580ccae0 2023-07-06T01:21:37.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.623+0000 7feb3ffff700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7feb4406ace0 0x7feb4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.623+0000 7feb3ffff700 1 -- 172.21.15.22:0/1578714734 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6147+0+0 (secure 0 0 0) 0x7feb480891f0 con 0x7feb580ccae0 2023-07-06T01:21:37.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.623+0000 7feb577fe700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7feb4406ace0 0x7feb4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.629 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.623+0000 7feb577fe700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7feb4406ace0 0x7feb4406d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feb5806ed00 tx=0x7feb4c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:37.630 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.627+0000 7feb3ffff700 1 -- 172.21.15.22:0/1578714734 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feb480542d0 con 0x7feb580ccae0 2023-07-06T01:21:37.829 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.828+0000 7feb5e141700 1 -- 172.21.15.22:0/1578714734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb580cddc0 con 0x7feb580ccae0 2023-07-06T01:21:37.830 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.828+0000 7feb3ffff700 1 -- 172.21.15.22:0/1578714734 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb480578f0 con 0x7feb580ccae0 2023-07-06T01:21:37.830 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:21:37.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.830+0000 7feb5e141700 1 -- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7feb4406ace0 msgr2=0x7feb4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.832 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.830+0000 7feb5e141700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7feb4406ace0 0x7feb4406d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feb5806ed00 tx=0x7feb4c008040 comp rx=0 tx=0).stop 2023-07-06T01:21:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.830+0000 7feb5e141700 1 -- 172.21.15.22:0/1578714734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb580ccae0 msgr2=0x7feb581f3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.830+0000 7feb5e141700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb580ccae0 0x7feb581f3ae0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7feb480011e0 tx=0x7feb4800cae0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.831+0000 7feb5e141700 1 -- 172.21.15.22:0/1578714734 shutdown_connections 2023-07-06T01:21:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.831+0000 7feb5e141700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb58159360 0x7feb5815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.831+0000 7feb5e141700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7feb4406ace0 0x7feb4406d190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.831+0000 7feb5e141700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb580cc660 0x7feb581f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.833 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.831+0000 7feb5e141700 1 --2- 172.21.15.22:0/1578714734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb580ccae0 0x7feb581f3ae0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.831+0000 7feb5e141700 1 -- 172.21.15.22:0/1578714734 >> 172.21.15.22:0/1578714734 conn(0x7feb580b1890 msgr2=0x7feb5815cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.831+0000 7feb5e141700 1 -- 172.21.15.22:0/1578714734 shutdown_connections 2023-07-06T01:21:37.834 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.831+0000 7feb5e141700 1 -- 172.21.15.22:0/1578714734 wait complete. 2023-07-06T01:21:37.844 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T01:21:37.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.970+0000 7f75c984f700 1 Processor -- start 2023-07-06T01:21:37.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.970+0000 7f75c984f700 1 -- start start 2023-07-06T01:21:37.973 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c984f700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75c406be00 0x7f75c406c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c984f700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f75c406c850 0x7f75c40cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c984f700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c4070410 con 0x7f75c40cdc90 2023-07-06T01:21:37.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c984f700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c406ccf0 con 0x7f75c406c850 2023-07-06T01:21:37.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c984f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c406b110 con 0x7f75c406be00 2023-07-06T01:21:37.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c2ffd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f75c40cdc90 0x7f75c406f950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38552/0 (socket says 172.21.15.22:38552) 2023-07-06T01:21:37.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c2ffd700 1 -- 172.21.15.22:0/345730736 learned_addr learned my addr 172.21.15.22:0/345730736 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:37.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c27fc700 1 --2- 172.21.15.22:0/345730736 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75c406be00 0x7f75c406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c37fe700 1 --2- 172.21.15.22:0/345730736 >> v2:172.21.15.103:3300/0 conn(0x7f75c406c850 0x7f75c40cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c1ffb700 1 -- 172.21.15.22:0/345730736 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226380573 0 0) 0x7f75c4070410 con 0x7f75c40cdc90 2023-07-06T01:21:37.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c1ffb700 1 -- 172.21.15.22:0/345730736 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75b00035b0 con 0x7f75c40cdc90 2023-07-06T01:21:37.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c27fc700 1 -- 172.21.15.22:0/345730736 >> v2:172.21.15.103:3300/0 conn(0x7f75c406c850 msgr2=0x7f75c40cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:37.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c27fc700 1 --2- 172.21.15.22:0/345730736 >> v2:172.21.15.103:3300/0 conn(0x7f75c406c850 0x7f75c40cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c27fc700 1 -- 172.21.15.22:0/345730736 >> v1:172.21.15.22:6789/0 conn(0x7f75c40cdc90 legacy=0x7f75c406f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.971+0000 7f75c27fc700 1 -- 172.21.15.22:0/345730736 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c406b320 con 0x7f75c406be00 2023-07-06T01:21:37.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.972+0000 7f75c27fc700 1 --2- 172.21.15.22:0/345730736 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75c406be00 0x7f75c406c310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009840 tx=0x7f75ac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1efa43a404f51c7b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:37.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.972+0000 7f75c1ffb700 1 -- 172.21.15.22:0/345730736 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac011830 con 0x7f75c406be00 2023-07-06T01:21:37.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.972+0000 7f75c1ffb700 1 -- 172.21.15.22:0/345730736 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ac011990 con 0x7f75c406be00 2023-07-06T01:21:37.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.972+0000 7f75c1ffb700 1 -- 172.21.15.22:0/345730736 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac011d20 con 0x7f75c406be00 2023-07-06T01:21:37.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.972+0000 7f75c984f700 1 -- 172.21.15.22:0/345730736 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75c406be00 msgr2=0x7f75c406c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.972+0000 7f75c984f700 1 --2- 172.21.15.22:0/345730736 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75c406be00 0x7f75c406c310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f75ac009840 tx=0x7f75ac009b50 comp rx=0 tx=0).stop 2023-07-06T01:21:37.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.972+0000 7f75c984f700 1 -- 172.21.15.22:0/345730736 shutdown_connections 2023-07-06T01:21:37.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.972+0000 7f75c984f700 1 --2- 172.21.15.22:0/345730736 >> v2:172.21.15.103:3300/0 conn(0x7f75c406c850 0x7f75c40cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.972+0000 7f75c984f700 1 --2- 172.21.15.22:0/345730736 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75c406be00 0x7f75c406c310 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.972+0000 7f75c984f700 1 -- 172.21.15.22:0/345730736 >> 172.21.15.22:0/345730736 conn(0x7f75c406dc30 msgr2=0x7f75c406e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.972+0000 7f75c984f700 1 -- 172.21.15.22:0/345730736 shutdown_connections 2023-07-06T01:21:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.972+0000 7f75c984f700 1 -- 172.21.15.22:0/345730736 wait complete. 2023-07-06T01:21:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.974+0000 7f75c984f700 1 Processor -- start 2023-07-06T01:21:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.974+0000 7f75c984f700 1 -- start start 2023-07-06T01:21:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.974+0000 7f75c984f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75c406be00 0x7f75c415e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.974+0000 7f75c984f700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75c406c850 0x7f75c40b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.974+0000 7f75c984f700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40cdc90 0x7f75c40cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.974+0000 7f75c984f700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c40b2990 con 0x7f75c40cdc90 2023-07-06T01:21:37.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.974+0000 7f75c984f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c40b1d10 con 0x7f75c406be00 2023-07-06T01:21:37.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.974+0000 7f75c984f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c40b1ff0 con 0x7f75c406c850 2023-07-06T01:21:37.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75c37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40cdc90 0x7f75c40cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75c37fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40cdc90 0x7f75c40cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47818/0 (socket says 172.21.15.22:47818) 2023-07-06T01:21:37.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75c37fe700 1 -- 172.21.15.22:0/2936808614 learned_addr learned my addr 172.21.15.22:0/2936808614 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:37.980 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75c2ffd700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75c406be00 0x7f75c415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75c27fc700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75c406c850 0x7f75c40b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75c37fe700 1 -- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75c406c850 msgr2=0x7f75c40b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75c37fe700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75c406c850 0x7f75c40b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75c37fe700 1 -- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75c406be00 msgr2=0x7f75c415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75c37fe700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75c406be00 0x7f75c415e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75c37fe700 1 -- 172.21.15.22:0/2936808614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac0094a0 con 0x7f75c40cdc90 2023-07-06T01:21:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75c37fe700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40cdc90 0x7f75c40cac10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00efc0 tx=0x7f75bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:37.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75bbfff700 1 -- 172.21.15.22:0/2936808614 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc00cd40 con 0x7f75c40cdc90 2023-07-06T01:21:37.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75bbfff700 1 -- 172.21.15.22:0/2936808614 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75bc00f040 con 0x7f75c40cdc90 2023-07-06T01:21:37.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75c984f700 1 -- 172.21.15.22:0/2936808614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c40b22d0 con 0x7f75c40cdc90 2023-07-06T01:21:37.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75bbfff700 1 -- 172.21.15.22:0/2936808614 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc0146c0 con 0x7f75c40cdc90 2023-07-06T01:21:37.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.975+0000 7f75c984f700 1 -- 172.21.15.22:0/2936808614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c415ecb0 con 0x7f75c40cdc90 2023-07-06T01:21:37.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.976+0000 7f75c984f700 1 -- 172.21.15.22:0/2936808614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c415eea0 con 0x7f75c40cdc90 2023-07-06T01:21:37.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.977+0000 7f75bbfff700 1 -- 172.21.15.22:0/2936808614 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f75bc014820 con 0x7f75c40cdc90 2023-07-06T01:21:37.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.977+0000 7f75bbfff700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f75a406ad10 0x7f75a406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:37.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.977+0000 7f75bbfff700 1 -- 172.21.15.22:0/2936808614 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6147+0+0 (secure 0 0 0) 0x7f75bc089e90 con 0x7f75c40cdc90 2023-07-06T01:21:37.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.977+0000 7f75c2ffd700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f75a406ad10 0x7f75a406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:37.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.978+0000 7f75c2ffd700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f75a406ad10 0x7f75a406d1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f75b40011e0 tx=0x7f75b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:37.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:37.982+0000 7f75bbfff700 1 -- 172.21.15.22:0/2936808614 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f75bc0588d0 con 0x7f75c40cdc90 2023-07-06T01:21:38.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.182+0000 7f75c984f700 1 -- 172.21.15.22:0/2936808614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f75c40b2ad0 con 0x7f75c40cdc90 2023-07-06T01:21:38.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.182+0000 7f75bbfff700 1 -- 172.21.15.22:0/2936808614 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f75bc018020 con 0x7f75c40cdc90 2023-07-06T01:21:38.184 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-07-06T01:21:38.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.185+0000 7f75c984f700 1 -- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f75a406ad10 msgr2=0x7f75a406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.185+0000 7f75c984f700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f75a406ad10 0x7f75a406d1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f75b40011e0 tx=0x7f75b4009040 comp rx=0 tx=0).stop 2023-07-06T01:21:38.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.185+0000 7f75c984f700 1 -- 172.21.15.22:0/2936808614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40cdc90 msgr2=0x7f75c40cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.185+0000 7f75c984f700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40cdc90 0x7f75c40cac10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00efc0 tx=0x7f75bc00c3a0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.185+0000 7f75c984f700 1 -- 172.21.15.22:0/2936808614 shutdown_connections 2023-07-06T01:21:38.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.185+0000 7f75c984f700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75c40cdc90 0x7f75c40cac10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.185+0000 7f75c984f700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f75a406ad10 0x7f75a406d1c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.185+0000 7f75c984f700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f75c406be00 0x7f75c415e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.185+0000 7f75c984f700 1 --2- 172.21.15.22:0/2936808614 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f75c406c850 0x7f75c40b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.185+0000 7f75c984f700 1 -- 172.21.15.22:0/2936808614 >> 172.21.15.22:0/2936808614 conn(0x7f75c406dc30 msgr2=0x7f75c40add10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:38.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.185+0000 7f75c984f700 1 -- 172.21.15.22:0/2936808614 shutdown_connections 2023-07-06T01:21:38.189 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.185+0000 7f75c984f700 1 -- 172.21.15.22:0/2936808614 wait complete. 2023-07-06T01:21:38.199 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T01:21:38.326 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.325+0000 7ff12d44c700 1 Processor -- start 2023-07-06T01:21:38.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.325+0000 7ff12d44c700 1 -- start start 2023-07-06T01:21:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.326+0000 7ff12d44c700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff12815a710 0x7ff1280ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.326+0000 7ff12d44c700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1280af270 con 0x7ff1280aecb0 2023-07-06T01:21:38.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.326+0000 7ff12d44c700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1280af550 con 0x7ff1280cef70 2023-07-06T01:21:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.326+0000 7ff12d44c700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1280af830 con 0x7ff12815a710 2023-07-06T01:21:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.326+0000 7ff1267fc700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff1280aecb0 0x7ff1280ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38568/0 (socket says 172.21.15.22:38568) 2023-07-06T01:21:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.326+0000 7ff1267fc700 1 -- 172.21.15.22:0/4029113380 learned_addr learned my addr 172.21.15.22:0/4029113380 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.326+0000 7ff126ffd700 1 --2- 172.21.15.22:0/4029113380 >> v2:172.21.15.103:3300/0 conn(0x7ff12815a710 0x7ff1280ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.326+0000 7ff125ffb700 1 -- 172.21.15.22:0/4029113380 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155749554 0 0) 0x7ff1280af270 con 0x7ff1280aecb0 2023-07-06T01:21:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.326+0000 7ff125ffb700 1 -- 172.21.15.22:0/4029113380 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1100035b0 con 0x7ff1280aecb0 2023-07-06T01:21:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.326+0000 7ff126ffd700 1 -- 172.21.15.22:0/4029113380 >> v1:172.21.15.103:6789/0 conn(0x7ff1280cef70 legacy=0x7ff1280cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.326+0000 7ff126ffd700 1 -- 172.21.15.22:0/4029113380 >> v1:172.21.15.22:6789/0 conn(0x7ff1280aecb0 legacy=0x7ff1280ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.326+0000 7ff126ffd700 1 -- 172.21.15.22:0/4029113380 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1280cb2e0 con 0x7ff12815a710 2023-07-06T01:21:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.326+0000 7ff126ffd700 1 --2- 172.21.15.22:0/4029113380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12815a710 0x7ff1280ae770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff118009840 tx=0x7ff118009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40740c4203d299ca server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.327+0000 7ff125ffb700 1 -- 172.21.15.22:0/4029113380 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118011830 con 0x7ff12815a710 2023-07-06T01:21:38.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.327+0000 7ff125ffb700 1 -- 172.21.15.22:0/4029113380 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff118011990 con 0x7ff12815a710 2023-07-06T01:21:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.327+0000 7ff125ffb700 1 -- 172.21.15.22:0/4029113380 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118011d20 con 0x7ff12815a710 2023-07-06T01:21:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.327+0000 7ff12d44c700 1 -- 172.21.15.22:0/4029113380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12815a710 msgr2=0x7ff1280ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.327+0000 7ff12d44c700 1 --2- 172.21.15.22:0/4029113380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12815a710 0x7ff1280ae770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff118009840 tx=0x7ff118009b50 comp rx=0 tx=0).stop 2023-07-06T01:21:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.327+0000 7ff12d44c700 1 -- 172.21.15.22:0/4029113380 shutdown_connections 2023-07-06T01:21:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.327+0000 7ff12d44c700 1 --2- 172.21.15.22:0/4029113380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff12815a710 0x7ff1280ae770 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.327+0000 7ff12d44c700 1 -- 172.21.15.22:0/4029113380 >> 172.21.15.22:0/4029113380 conn(0x7ff1280999f0 msgr2=0x7ff128099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:38.333 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.327+0000 7ff12d44c700 1 -- 172.21.15.22:0/4029113380 shutdown_connections 2023-07-06T01:21:38.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.327+0000 7ff12d44c700 1 -- 172.21.15.22:0/4029113380 wait complete. 2023-07-06T01:21:38.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.329+0000 7ff12d44c700 1 Processor -- start 2023-07-06T01:21:38.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.329+0000 7ff12d44c700 1 -- start start 2023-07-06T01:21:38.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.329+0000 7ff12d44c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280aecb0 0x7ff1280b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:38.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.329+0000 7ff12d44c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1280cef70 0x7ff1280c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:38.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.329+0000 7ff12d44c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff12815a710 0x7ff128153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:38.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.329+0000 7ff12d44c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12806c850 con 0x7ff1280aecb0 2023-07-06T01:21:38.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.329+0000 7ff12d44c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12806cb30 con 0x7ff1280cef70 2023-07-06T01:21:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.329+0000 7ff12d44c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280b5fb0 con 0x7ff12815a710 2023-07-06T01:21:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.329+0000 7ff126ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280aecb0 0x7ff1280b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff126ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280aecb0 0x7ff1280b67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47828/0 (socket says 172.21.15.22:47828) 2023-07-06T01:21:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff126ffd700 1 -- 172.21.15.22:0/2933424211 learned_addr learned my addr 172.21.15.22:0/2933424211 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff1267fc700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1280cef70 0x7ff1280c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff1277fe700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff12815a710 0x7ff128153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:38.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff126ffd700 1 -- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff12815a710 msgr2=0x7ff128153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff126ffd700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff12815a710 0x7ff128153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff126ffd700 1 -- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1280cef70 msgr2=0x7ff1280c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff126ffd700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1280cef70 0x7ff1280c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff126ffd700 1 -- 172.21.15.22:0/2933424211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1180094a0 con 0x7ff1280aecb0 2023-07-06T01:21:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff126ffd700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280aecb0 0x7ff1280b67b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff118009b90 tx=0x7ff118004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2933424211 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1180036a0 con 0x7ff1280aecb0 2023-07-06T01:21:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2933424211 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11801e070 con 0x7ff1280aecb0 2023-07-06T01:21:38.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2933424211 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118003a60 con 0x7ff1280aecb0 2023-07-06T01:21:38.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff12d44c700 1 -- 172.21.15.22:0/2933424211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1280b6290 con 0x7ff1280aecb0 2023-07-06T01:21:38.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.330+0000 7ff12d44c700 1 -- 172.21.15.22:0/2933424211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1280b5310 con 0x7ff1280aecb0 2023-07-06T01:21:38.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.331+0000 7ff12d44c700 1 -- 172.21.15.22:0/2933424211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12806ccf0 con 0x7ff1280aecb0 2023-07-06T01:21:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.332+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2933424211 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff118003bc0 con 0x7ff1280aecb0 2023-07-06T01:21:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.332+0000 7ff10f7fe700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff11406ace0 0x7ff11406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.332+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2933424211 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6147+0+0 (secure 0 0 0) 0x7ff11808a400 con 0x7ff1280aecb0 2023-07-06T01:21:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.332+0000 7ff1267fc700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff11406ace0 0x7ff11406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:38.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.333+0000 7ff1267fc700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff11406ace0 0x7ff11406d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff1280b3010 tx=0x7ff11c007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:38.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.338+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2933424211 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff118055440 con 0x7ff1280aecb0 2023-07-06T01:21:38.536 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.535+0000 7ff12d44c700 1 -- 172.21.15.22:0/2933424211 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff1280b64a0 con 0x7ff1280aecb0 2023-07-06T01:21:38.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.535+0000 7ff10f7fe700 1 -- 172.21.15.22:0/2933424211 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff118058a60 con 0x7ff1280aecb0 2023-07-06T01:21:38.538 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-07-06T01:21:38.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.537+0000 7ff12d44c700 1 -- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff11406ace0 msgr2=0x7ff11406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.537+0000 7ff12d44c700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff11406ace0 0x7ff11406d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff1280b3010 tx=0x7ff11c007dc0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.537+0000 7ff12d44c700 1 -- 172.21.15.22:0/2933424211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280aecb0 msgr2=0x7ff1280b67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.537+0000 7ff12d44c700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280aecb0 0x7ff1280b67b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff118009b90 tx=0x7ff118004750 comp rx=0 tx=0).stop 2023-07-06T01:21:38.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.537+0000 7ff12d44c700 1 -- 172.21.15.22:0/2933424211 shutdown_connections 2023-07-06T01:21:38.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.537+0000 7ff12d44c700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff12815a710 0x7ff128153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.537+0000 7ff12d44c700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1280aecb0 0x7ff1280b67b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.537+0000 7ff12d44c700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff11406ace0 0x7ff11406d190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.537+0000 7ff12d44c700 1 --2- 172.21.15.22:0/2933424211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1280cef70 0x7ff1280c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.537+0000 7ff12d44c700 1 -- 172.21.15.22:0/2933424211 >> 172.21.15.22:0/2933424211 conn(0x7ff1280999f0 msgr2=0x7ff1280cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:38.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.537+0000 7ff12d44c700 1 -- 172.21.15.22:0/2933424211 shutdown_connections 2023-07-06T01:21:38.541 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.537+0000 7ff12d44c700 1 -- 172.21.15.22:0/2933424211 wait complete. 2023-07-06T01:21:38.551 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T01:21:38.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.676+0000 7f47408ee700 1 Processor -- start 2023-07-06T01:21:38.678 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.676+0000 7f47408ee700 1 -- start start 2023-07-06T01:21:38.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.677+0000 7f47408ee700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f473c0c9bb0 0x7f473c06c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:38.680 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.677+0000 7f47408ee700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f473c06c6d0 0x7f473c070800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:38.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.677+0000 7f47408ee700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473c06cb10 con 0x7f473c158c90 2023-07-06T01:21:38.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.677+0000 7f47408ee700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473c06a710 con 0x7f473c0c9bb0 2023-07-06T01:21:38.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.677+0000 7f47408ee700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c06b1a0 con 0x7f473c06c6d0 2023-07-06T01:21:38.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.677+0000 7f4739d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f473c0c9bb0 0x7f473c06c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:38.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.677+0000 7f4739d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f473c0c9bb0 0x7f473c06c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47838/0 (socket says 172.21.15.22:47838) 2023-07-06T01:21:38.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.677+0000 7f4739d9b700 1 -- 172.21.15.22:0/327943082 learned_addr learned my addr 172.21.15.22:0/327943082 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:38.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.678+0000 7f473ad9d700 1 --2- 172.21.15.22:0/327943082 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f473c06c6d0 0x7f473c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:38.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.678+0000 7f4739d9b700 1 -- 172.21.15.22:0/327943082 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f473c06c6d0 msgr2=0x7f473c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:38.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.678+0000 7f4739d9b700 1 --2- 172.21.15.22:0/327943082 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f473c06c6d0 0x7f473c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.678+0000 7f4739d9b700 1 -- 172.21.15.22:0/327943082 >> v1:172.21.15.103:6789/0 conn(0x7f473c158c90 legacy=0x7f473c06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.678+0000 7f4739d9b700 1 -- 172.21.15.22:0/327943082 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c070320 con 0x7f473c0c9bb0 2023-07-06T01:21:38.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.678+0000 7f473ad9d700 1 --2- 172.21.15.22:0/327943082 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f473c06c6d0 0x7f473c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:38.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.678+0000 7f4739d9b700 1 --2- 172.21.15.22:0/327943082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f473c0c9bb0 0x7f473c06c190 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4730009920 tx=0x7f4730009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ef035047cb8908 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:38.682 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.678+0000 7f473959a700 1 -- 172.21.15.22:0/327943082 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473000ba20 con 0x7f473c0c9bb0 2023-07-06T01:21:38.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.678+0000 7f473959a700 1 -- 172.21.15.22:0/327943082 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473000bb80 con 0x7f473c0c9bb0 2023-07-06T01:21:38.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.678+0000 7f473959a700 1 -- 172.21.15.22:0/327943082 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730009e70 con 0x7f473c0c9bb0 2023-07-06T01:21:38.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.679+0000 7f47408ee700 1 -- 172.21.15.22:0/327943082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f473c0c9bb0 msgr2=0x7f473c06c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.679+0000 7f47408ee700 1 --2- 172.21.15.22:0/327943082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f473c0c9bb0 0x7f473c06c190 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4730009920 tx=0x7f4730009c30 comp rx=0 tx=0).stop 2023-07-06T01:21:38.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.679+0000 7f47408ee700 1 -- 172.21.15.22:0/327943082 shutdown_connections 2023-07-06T01:21:38.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.679+0000 7f47408ee700 1 --2- 172.21.15.22:0/327943082 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f473c06c6d0 0x7f473c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.679+0000 7f47408ee700 1 --2- 172.21.15.22:0/327943082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f473c0c9bb0 0x7f473c06c190 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.679+0000 7f47408ee700 1 -- 172.21.15.22:0/327943082 >> 172.21.15.22:0/327943082 conn(0x7f473c0ccae0 msgr2=0x7f473c0cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:38.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.679+0000 7f47408ee700 1 -- 172.21.15.22:0/327943082 shutdown_connections 2023-07-06T01:21:38.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.679+0000 7f47408ee700 1 -- 172.21.15.22:0/327943082 wait complete. 2023-07-06T01:21:38.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.680+0000 7f47408ee700 1 Processor -- start 2023-07-06T01:21:38.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.680+0000 7f47408ee700 1 -- start start 2023-07-06T01:21:38.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f47408ee700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f473c06c6d0 0x7f473c159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:38.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f47408ee700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f473c0c9bb0 0x7f473c15c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:38.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f47408ee700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f473c158c90 0x7f473c15cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:38.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f47408ee700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0c8730 con 0x7f473c06c6d0 2023-07-06T01:21:38.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f47408ee700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c1599b0 con 0x7f473c0c9bb0 2023-07-06T01:21:38.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f47408ee700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0c7e20 con 0x7f473c158c90 2023-07-06T01:21:38.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f473a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f473c06c6d0 0x7f473c159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:38.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f473a59c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f473c06c6d0 0x7f473c159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47842/0 (socket says 172.21.15.22:47842) 2023-07-06T01:21:38.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f473a59c700 1 -- 172.21.15.22:0/3345280644 learned_addr learned my addr 172.21.15.22:0/3345280644 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:38.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f4739d9b700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f473c0c9bb0 0x7f473c15c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:38.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f473ad9d700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f473c158c90 0x7f473c15cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:38.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f473a59c700 1 -- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f473c158c90 msgr2=0x7f473c15cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f473a59c700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f473c158c90 0x7f473c15cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f473a59c700 1 -- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f473c0c9bb0 msgr2=0x7f473c15c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f473a59c700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f473c0c9bb0 0x7f473c15c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f473a59c700 1 -- 172.21.15.22:0/3345280644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4730009550 con 0x7f473c06c6d0 2023-07-06T01:21:38.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.681+0000 7f473ad9d700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f473c158c90 0x7f473c15cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:38.688 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.682+0000 7f473a59c700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f473c06c6d0 0x7f473c159180 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f472c00c450 tx=0x7f472c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:38.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.682+0000 7f472affd700 1 -- 172.21.15.22:0/3345280644 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c0052d0 con 0x7f473c06c6d0 2023-07-06T01:21:38.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.682+0000 7f472affd700 1 -- 172.21.15.22:0/3345280644 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c005430 con 0x7f473c06c6d0 2023-07-06T01:21:38.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.682+0000 7f47408ee700 1 -- 172.21.15.22:0/3345280644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c0c80e0 con 0x7f473c06c6d0 2023-07-06T01:21:38.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.682+0000 7f472affd700 1 -- 172.21.15.22:0/3345280644 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c0038c0 con 0x7f473c06c6d0 2023-07-06T01:21:38.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.682+0000 7f47408ee700 1 -- 172.21.15.22:0/3345280644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c1596c0 con 0x7f473c06c6d0 2023-07-06T01:21:38.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.683+0000 7f47408ee700 1 -- 172.21.15.22:0/3345280644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473c0744b0 con 0x7f473c06c6d0 2023-07-06T01:21:38.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.683+0000 7f472affd700 1 -- 172.21.15.22:0/3345280644 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f472c003a20 con 0x7f473c06c6d0 2023-07-06T01:21:38.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.684+0000 7f472affd700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f471c06ace0 0x7f471c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:38.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.684+0000 7f472affd700 1 -- 172.21.15.22:0/3345280644 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6147+0+0 (secure 0 0 0) 0x7f472c088880 con 0x7f473c06c6d0 2023-07-06T01:21:38.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.684+0000 7f4739d9b700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f471c06ace0 0x7f471c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:38.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.684+0000 7f4739d9b700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f471c06ace0 0x7f471c06d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4730000c00 tx=0x7f4730008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:38.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.688+0000 7f472affd700 1 -- 172.21.15.22:0/3345280644 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f472c0538d0 con 0x7f473c06c6d0 2023-07-06T01:21:38.904 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.903+0000 7f47408ee700 1 -- 172.21.15.22:0/3345280644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f473c0b5350 con 0x7f473c06c6d0 2023-07-06T01:21:38.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.903+0000 7f472affd700 1 -- 172.21.15.22:0/3345280644 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f472c056ef0 con 0x7f473c06c6d0 2023-07-06T01:21:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.905+0000 7f47408ee700 1 -- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f471c06ace0 msgr2=0x7f471c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.905+0000 7f47408ee700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f471c06ace0 0x7f471c06d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4730000c00 tx=0x7f4730008030 comp rx=0 tx=0).stop 2023-07-06T01:21:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.905+0000 7f47408ee700 1 -- 172.21.15.22:0/3345280644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f473c06c6d0 msgr2=0x7f473c159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.905+0000 7f47408ee700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f473c06c6d0 0x7f473c159180 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f472c00c450 tx=0x7f472c00c810 comp rx=0 tx=0).stop 2023-07-06T01:21:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.905+0000 7f47408ee700 1 -- 172.21.15.22:0/3345280644 shutdown_connections 2023-07-06T01:21:38.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.905+0000 7f47408ee700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f473c158c90 0x7f473c15cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.905+0000 7f47408ee700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f473c06c6d0 0x7f473c159180 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.905+0000 7f47408ee700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f471c06ace0 0x7f471c06d190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.905+0000 7f47408ee700 1 --2- 172.21.15.22:0/3345280644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f473c0c9bb0 0x7f473c15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:38.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.905+0000 7f47408ee700 1 -- 172.21.15.22:0/3345280644 >> 172.21.15.22:0/3345280644 conn(0x7f473c0ccae0 msgr2=0x7f473c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:38.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.905+0000 7f47408ee700 1 -- 172.21.15.22:0/3345280644 shutdown_connections 2023-07-06T01:21:38.908 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:38.905+0000 7f47408ee700 1 -- 172.21.15.22:0/3345280644 wait complete. 2023-07-06T01:21:38.909 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2023-07-06T01:21:38.919 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-06T01:21:38.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:38.920 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-07-06T01:21:39.124 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:21:39.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:39.125 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T01:21:39.166 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:21:39.166 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:21:39.184 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284276k,nr_inodes=4071069,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19160 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T01:21:39.185 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:21:39.185 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:21:39.223 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19513 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T01:21:39.224 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:21:39.224 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:21:39.242 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284276k,nr_inodes=4071069,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19160 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T01:21:39.242 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:21:39.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.387+0000 7effea0d2700 1 Processor -- start 2023-07-06T01:21:39.389 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.387+0000 7effea0d2700 1 -- start start 2023-07-06T01:21:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.388+0000 7effea0d2700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40af2a0 0x7effe406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.388+0000 7effea0d2700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe40af670 con 0x7effe406a000 2023-07-06T01:21:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.388+0000 7effea0d2700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe406d030 con 0x7effe40cc320 2023-07-06T01:21:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.388+0000 7effea0d2700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe40cc7a0 con 0x7effe40af2a0 2023-07-06T01:21:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.388+0000 7effe2ffd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7effe406a000 0x7effe40cbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38596/0 (socket says 172.21.15.22:38596) 2023-07-06T01:21:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.388+0000 7effe2ffd700 1 -- 172.21.15.22:0/1507727875 learned_addr learned my addr 172.21.15.22:0/1507727875 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:39.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.388+0000 7effe37fe700 1 --2- 172.21.15.22:0/1507727875 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40af2a0 0x7effe406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.388+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455388061 0 0) 0x7effe40af670 con 0x7effe406a000 2023-07-06T01:21:39.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.388+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effd00035b0 con 0x7effe406a000 2023-07-06T01:21:39.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926730301 0 0) 0x7effe406d030 con 0x7effe40cc320 2023-07-06T01:21:39.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effe40af670 con 0x7effe40cc320 2023-07-06T01:21:39.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2558611374 0 0) 0x7effd00035b0 con 0x7effe406a000 2023-07-06T01:21:39.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40af2a0 msgr2=0x7effe406caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:39.393 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 --2- 172.21.15.22:0/1507727875 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40af2a0 0x7effe406caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 >> v1:172.21.15.103:6789/0 conn(0x7effe40cc320 legacy=0x7effe4160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe40cd560 con 0x7effe406a000 2023-07-06T01:21:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7effcc001670 con 0x7effe406a000 2023-07-06T01:21:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe37fe700 1 --2- 172.21.15.22:0/1507727875 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40af2a0 0x7effe406caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 >> v1:172.21.15.22:6789/0 conn(0x7effe406a000 legacy=0x7effe40cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 --2- 172.21.15.22:0/1507727875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effd0003bc0 0x7effd0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.394 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 --2- 172.21.15.22:0/1507727875 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40af2a0 0x7effd0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 --2- 172.21.15.22:0/1507727875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effd0004d90 0x7effd0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe40cc7a0 con 0x7effd0003bc0 2023-07-06T01:21:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd000bac0 con 0x7effd0004d90 2023-07-06T01:21:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd000bc30 con 0x7effe40af2a0 2023-07-06T01:21:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe37fe700 1 --2- 172.21.15.22:0/1507727875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effd0003bc0 0x7effd0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe2ffd700 1 --2- 172.21.15.22:0/1507727875 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40af2a0 0x7effd0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe3fff700 1 --2- 172.21.15.22:0/1507727875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effd0004d90 0x7effd0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe37fe700 1 -- 172.21.15.22:0/1507727875 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40af2a0 msgr2=0x7effd0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe37fe700 1 --2- 172.21.15.22:0/1507727875 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40af2a0 0x7effd0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe37fe700 1 -- 172.21.15.22:0/1507727875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effd0004d90 msgr2=0x7effd0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe37fe700 1 --2- 172.21.15.22:0/1507727875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effd0004d90 0x7effd0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe37fe700 1 -- 172.21.15.22:0/1507727875 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd40094a0 con 0x7effd0003bc0 2023-07-06T01:21:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe37fe700 1 --2- 172.21.15.22:0/1507727875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effd0003bc0 0x7effd0003fb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7effd4009d30 tx=0x7effd0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e26b4ac85a5e689 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:39.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.389+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4009e80 con 0x7effd0003bc0 2023-07-06T01:21:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.390+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd400e940 con 0x7effd0003bc0 2023-07-06T01:21:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.390+0000 7effe27fc700 1 -- 172.21.15.22:0/1507727875 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd400ecb0 con 0x7effd0003bc0 2023-07-06T01:21:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.390+0000 7effea0d2700 1 -- 172.21.15.22:0/1507727875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effd0003bc0 msgr2=0x7effd0003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.390+0000 7effea0d2700 1 --2- 172.21.15.22:0/1507727875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effd0003bc0 0x7effd0003fb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7effd4009d30 tx=0x7effd0005880 comp rx=0 tx=0).stop 2023-07-06T01:21:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.390+0000 7effe3fff700 1 -- 172.21.15.22:0/1507727875 reap_dead start 2023-07-06T01:21:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.390+0000 7effea0d2700 1 -- 172.21.15.22:0/1507727875 shutdown_connections 2023-07-06T01:21:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.390+0000 7effea0d2700 1 -- 172.21.15.22:0/1507727875 >> 172.21.15.22:0/1507727875 conn(0x7effe40b1890 msgr2=0x7effe40b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:39.397 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.390+0000 7effea0d2700 1 -- 172.21.15.22:0/1507727875 shutdown_connections 2023-07-06T01:21:39.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.390+0000 7effea0d2700 1 -- 172.21.15.22:0/1507727875 wait complete. 2023-07-06T01:21:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.391+0000 7effea0d2700 1 Processor -- start 2023-07-06T01:21:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.391+0000 7effea0d2700 1 -- start start 2023-07-06T01:21:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.392+0000 7effea0d2700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effd0003bc0 0x7effe41513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.392+0000 7effea0d2700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40b2b90 0x7effe40b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.392+0000 7effea0d2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effe40b3540 0x7effe40c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.392+0000 7effea0d2700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe40c6de0 con 0x7effe40b3540 2023-07-06T01:21:39.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.392+0000 7effea0d2700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe40b3e80 con 0x7effd0003bc0 2023-07-06T01:21:39.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.392+0000 7effea0d2700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe40b4160 con 0x7effe40b2b90 2023-07-06T01:21:39.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.392+0000 7effe3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effe40b3540 0x7effe40c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.392+0000 7effe3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effe40b3540 0x7effe40c7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47866/0 (socket says 172.21.15.22:47866) 2023-07-06T01:21:39.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.392+0000 7effe3fff700 1 -- 172.21.15.22:0/2658962402 learned_addr learned my addr 172.21.15.22:0/2658962402 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:39.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.392+0000 7effe37fe700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effd0003bc0 0x7effe41513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.392+0000 7effe2ffd700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40b2b90 0x7effe40b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.393+0000 7effe3fff700 1 -- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40b2b90 msgr2=0x7effe40b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:39.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.393+0000 7effe3fff700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40b2b90 0x7effe40b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.393+0000 7effe3fff700 1 -- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effd0003bc0 msgr2=0x7effe41513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:39.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.393+0000 7effe3fff700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effd0003bc0 0x7effe41513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.393+0000 7effe3fff700 1 -- 172.21.15.22:0/2658962402 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe40b4440 con 0x7effe40b3540 2023-07-06T01:21:39.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.393+0000 7effe2ffd700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40b2b90 0x7effe40b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:39.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.393+0000 7effe3fff700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effe40b3540 0x7effe40c7200 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7effe40cc6f0 tx=0x7effdc00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:39.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.393+0000 7effdbfff700 1 -- 172.21.15.22:0/2658962402 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc003c10 con 0x7effe40b3540 2023-07-06T01:21:39.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.393+0000 7effdbfff700 1 -- 172.21.15.22:0/2658962402 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effdc003d70 con 0x7effe40b3540 2023-07-06T01:21:39.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.393+0000 7effea0d2700 1 -- 172.21.15.22:0/2658962402 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe40b4720 con 0x7effe40b3540 2023-07-06T01:21:39.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.393+0000 7effdbfff700 1 -- 172.21.15.22:0/2658962402 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc00dbd0 con 0x7effe40b3540 2023-07-06T01:21:39.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.393+0000 7effea0d2700 1 -- 172.21.15.22:0/2658962402 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe40b39b0 con 0x7effe40b3540 2023-07-06T01:21:39.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.394+0000 7effea0d2700 1 -- 172.21.15.22:0/2658962402 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe40c6f20 con 0x7effe40b3540 2023-07-06T01:21:39.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.394+0000 7effdbfff700 1 -- 172.21.15.22:0/2658962402 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7effdc00dd30 con 0x7effe40b3540 2023-07-06T01:21:39.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.395+0000 7effdbfff700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7effc406ace0 0x7effc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.395+0000 7effdbfff700 1 -- 172.21.15.22:0/2658962402 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6147+0+0 (secure 0 0 0) 0x7effdc089840 con 0x7effe40b3540 2023-07-06T01:21:39.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.395+0000 7effe37fe700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7effc406ace0 0x7effc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.395+0000 7effe37fe700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7effc406ace0 0x7effc406d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7effd40090b0 tx=0x7effd401f040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:39.404 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.399+0000 7effdbfff700 1 -- 172.21.15.22:0/2658962402 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7effe40c6f20 con 0x7effe40b3540 2023-07-06T01:21:39.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.641+0000 7effea0d2700 1 -- 172.21.15.22:0/2658962402 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effe40c6f20 con 0x7effe40b3540 2023-07-06T01:21:39.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.641+0000 7effdbfff700 1 -- 172.21.15.22:0/2658962402 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4672 (secure 0 0 0) 0x7effe40c6f20 con 0x7effe40b3540 2023-07-06T01:21:39.647 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:21:39.648 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/219695916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":219695916},{"type":"v1","addr":"172.21.15.103:6836","nonce":219695916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3102932229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":3102932229},{"type":"v1","addr":"172.21.15.103:6837","nonce":3102932229}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3787686782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3787686782},{"type":"v1","addr":"172.21.15.22:6834","nonce":3787686782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-07-06T01:20:24.257974+0000","modified":"2023-07-06T01:21:37.327615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":22,"addr":"172.21.15.22:6835/612827809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":612827809},{"type":"v1","addr":"172.21.15.22:6835","nonce":612827809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T01:21:39.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.643+0000 7effea0d2700 1 -- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7effc406ace0 msgr2=0x7effc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:39.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.643+0000 7effea0d2700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7effc406ace0 0x7effc406d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7effd40090b0 tx=0x7effd401f040 comp rx=0 tx=0).stop 2023-07-06T01:21:39.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.643+0000 7effea0d2700 1 -- 172.21.15.22:0/2658962402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effe40b3540 msgr2=0x7effe40c7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:39.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.643+0000 7effea0d2700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effe40b3540 0x7effe40c7200 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7effe40cc6f0 tx=0x7effdc00d430 comp rx=0 tx=0).stop 2023-07-06T01:21:39.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.643+0000 7effea0d2700 1 -- 172.21.15.22:0/2658962402 shutdown_connections 2023-07-06T01:21:39.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.643+0000 7effea0d2700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7effe40b3540 0x7effe40c7200 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.643+0000 7effea0d2700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7effc406ace0 0x7effc406d190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.643+0000 7effea0d2700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effd0003bc0 0x7effe41513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.643+0000 7effea0d2700 1 --2- 172.21.15.22:0/2658962402 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effe40b2b90 0x7effe40b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.643+0000 7effea0d2700 1 -- 172.21.15.22:0/2658962402 >> 172.21.15.22:0/2658962402 conn(0x7effe40b1890 msgr2=0x7effe40ae0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:39.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.643+0000 7effea0d2700 1 -- 172.21.15.22:0/2658962402 shutdown_connections 2023-07-06T01:21:39.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.643+0000 7effea0d2700 1 -- 172.21.15.22:0/2658962402 wait complete. 2023-07-06T01:21:39.650 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2023-07-06T01:21:39.657 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:21:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.780+0000 7f321a2a9700 1 Processor -- start 2023-07-06T01:21:39.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.780+0000 7f321a2a9700 1 -- start start 2023-07-06T01:21:39.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f321a2a9700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f321406a710 0x7f32140cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f321a2a9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f321415f100 0x7f32140d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f321a2a9700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321415e3d0 con 0x7f321415a780 2023-07-06T01:21:39.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f321a2a9700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32140cc4b0 con 0x7f321406a710 2023-07-06T01:21:39.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f321a2a9700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321415ec70 con 0x7f321415f100 2023-07-06T01:21:39.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f32137fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f321406a710 0x7f32140cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f32137fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f321406a710 0x7f32140cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47872/0 (socket says 172.21.15.22:47872) 2023-07-06T01:21:39.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f32137fe700 1 -- 172.21.15.22:0/291286918 learned_addr learned my addr 172.21.15.22:0/291286918 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:39.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f3213fff700 1 --1- 172.21.15.22:0/291286918 >> v1:172.21.15.22:6789/0 conn(0x7f321415a780 0x7f321406a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38614/0 (socket says 172.21.15.22:38614) 2023-07-06T01:21:39.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f3218846700 1 --2- 172.21.15.22:0/291286918 >> v2:172.21.15.103:3300/0 conn(0x7f321415f100 0x7f32140d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f32137fe700 1 -- 172.21.15.22:0/291286918 >> v2:172.21.15.103:3300/0 conn(0x7f321415f100 msgr2=0x7f32140d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:39.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f32137fe700 1 --2- 172.21.15.22:0/291286918 >> v2:172.21.15.103:3300/0 conn(0x7f321415f100 0x7f32140d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f32137fe700 1 -- 172.21.15.22:0/291286918 >> v1:172.21.15.22:6789/0 conn(0x7f321415a780 legacy=0x7f321406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:39.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f32137fe700 1 -- 172.21.15.22:0/291286918 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32140d0670 con 0x7f321406a710 2023-07-06T01:21:39.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f3212ffd700 1 -- 172.21.15.22:0/291286918 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217824 0 0) 0x7f321415e3d0 con 0x7f321415a780 2023-07-06T01:21:39.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.781+0000 7f3218846700 1 --2- 172.21.15.22:0/291286918 >> v2:172.21.15.103:3300/0 conn(0x7f321415f100 0x7f32140d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:39.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.782+0000 7f32137fe700 1 --2- 172.21.15.22:0/291286918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f321406a710 0x7f32140cfbf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f32140cb270 tx=0x7f3204009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9da3961048c304 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:39.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.782+0000 7f3212ffd700 1 -- 172.21.15.22:0/291286918 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320400b870 con 0x7f321406a710 2023-07-06T01:21:39.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.782+0000 7f3212ffd700 1 -- 172.21.15.22:0/291286918 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320400b9d0 con 0x7f321406a710 2023-07-06T01:21:39.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.782+0000 7f3212ffd700 1 -- 172.21.15.22:0/291286918 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320400bd60 con 0x7f321406a710 2023-07-06T01:21:39.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.782+0000 7f321a2a9700 1 -- 172.21.15.22:0/291286918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f321406a710 msgr2=0x7f32140cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:39.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.782+0000 7f321a2a9700 1 --2- 172.21.15.22:0/291286918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f321406a710 0x7f32140cfbf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f32140cb270 tx=0x7f3204009bd0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.782+0000 7f321a2a9700 1 -- 172.21.15.22:0/291286918 shutdown_connections 2023-07-06T01:21:39.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.782+0000 7f321a2a9700 1 --2- 172.21.15.22:0/291286918 >> v2:172.21.15.103:3300/0 conn(0x7f321415f100 0x7f32140d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.782+0000 7f321a2a9700 1 --2- 172.21.15.22:0/291286918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f321406a710 0x7f32140cfbf0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.782+0000 7f321a2a9700 1 -- 172.21.15.22:0/291286918 >> 172.21.15.22:0/291286918 conn(0x7f321406b110 msgr2=0x7f321406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:39.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.782+0000 7f321a2a9700 1 -- 172.21.15.22:0/291286918 shutdown_connections 2023-07-06T01:21:39.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.782+0000 7f321a2a9700 1 -- 172.21.15.22:0/291286918 wait complete. 2023-07-06T01:21:39.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.783+0000 7f321a2a9700 1 Processor -- start 2023-07-06T01:21:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.784+0000 7f321a2a9700 1 -- start start 2023-07-06T01:21:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.784+0000 7f321a2a9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f321406a710 0x7f32140c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.784+0000 7f321a2a9700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f321415a780 0x7f32141f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.784+0000 7f321a2a9700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f321415f100 0x7f321415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.784+0000 7f321a2a9700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32140c6d30 con 0x7f321415f100 2023-07-06T01:21:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.784+0000 7f321a2a9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321415bdb0 con 0x7f321406a710 2023-07-06T01:21:39.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.784+0000 7f321a2a9700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32140c6ff0 con 0x7f321415a780 2023-07-06T01:21:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.784+0000 7f3218846700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f321415f100 0x7f321415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.784+0000 7f3218846700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f321415f100 0x7f321415b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47880/0 (socket says 172.21.15.22:47880) 2023-07-06T01:21:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f3218846700 1 -- 172.21.15.22:0/1511225122 learned_addr learned my addr 172.21.15.22:0/1511225122 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f32137fe700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f321415a780 0x7f32141f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f3213fff700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f321406a710 0x7f32140c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f3218846700 1 -- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f321415a780 msgr2=0x7f32141f05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f3218846700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f321415a780 0x7f32141f05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f3218846700 1 -- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f321406a710 msgr2=0x7f32140c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f3218846700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f321406a710 0x7f32140c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f3218846700 1 -- 172.21.15.22:0/1511225122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3204009550 con 0x7f321415f100 2023-07-06T01:21:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f3218846700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f321415f100 0x7f321415b560 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f320c00ce80 tx=0x7f320c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f3210ff9700 1 -- 172.21.15.22:0/1511225122 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c00ae20 con 0x7f321415f100 2023-07-06T01:21:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f3210ff9700 1 -- 172.21.15.22:0/1511225122 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320c015440 con 0x7f321415f100 2023-07-06T01:21:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f321a2a9700 1 -- 172.21.15.22:0/1511225122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32140c72d0 con 0x7f321415f100 2023-07-06T01:21:39.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f3210ff9700 1 -- 172.21.15.22:0/1511225122 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c0157f0 con 0x7f321415f100 2023-07-06T01:21:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.785+0000 7f321a2a9700 1 -- 172.21.15.22:0/1511225122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32141f0ae0 con 0x7f321415f100 2023-07-06T01:21:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.786+0000 7f321a2a9700 1 -- 172.21.15.22:0/1511225122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32140c77f0 con 0x7f321415f100 2023-07-06T01:21:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.787+0000 7f3210ff9700 1 -- 172.21.15.22:0/1511225122 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f320c00d430 con 0x7f321415f100 2023-07-06T01:21:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.787+0000 7f3210ff9700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f31f406ace0 0x7f31f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.787+0000 7f3210ff9700 1 -- 172.21.15.22:0/1511225122 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6147+0+0 (secure 0 0 0) 0x7f320c089b60 con 0x7f321415f100 2023-07-06T01:21:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.787+0000 7f3213fff700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f31f406ace0 0x7f31f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:39.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.788+0000 7f3213fff700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f31f406ace0 0x7f31f406d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f31fc000f60 tx=0x7f31fc007be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:39.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.792+0000 7f3210ff9700 1 -- 172.21.15.22:0/1511225122 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f32140c77f0 con 0x7f321415f100 2023-07-06T01:21:39.992 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.991+0000 7f321a2a9700 1 -- 172.21.15.22:0/1511225122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32140c77f0 con 0x7f321415f100 2023-07-06T01:21:39.993 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.992+0000 7f3210ff9700 1 -- 172.21.15.22:0/1511225122 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18382 (secure 0 0 0) 0x7f32140c77f0 con 0x7f321415f100 2023-07-06T01:21:39.994 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:21:39.995 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":37,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:21:36.323438+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:20:22.043370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T01:20:23.225298+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:21:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.994+0000 7f321a2a9700 1 -- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f31f406ace0 msgr2=0x7f31f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.994+0000 7f321a2a9700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f31f406ace0 0x7f31f406d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f31fc000f60 tx=0x7f31fc007be0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.994+0000 7f321a2a9700 1 -- 172.21.15.22:0/1511225122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f321415f100 msgr2=0x7f321415b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.994+0000 7f321a2a9700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f321415f100 0x7f321415b560 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f320c00ce80 tx=0x7f320c00cf60 comp rx=0 tx=0).stop 2023-07-06T01:21:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.994+0000 7f321a2a9700 1 -- 172.21.15.22:0/1511225122 shutdown_connections 2023-07-06T01:21:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.994+0000 7f321a2a9700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f321415f100 0x7f321415b560 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.994+0000 7f321a2a9700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f31f406ace0 0x7f31f406d190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.998 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.994+0000 7f321a2a9700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f321406a710 0x7f32140c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.994+0000 7f321a2a9700 1 --2- 172.21.15.22:0/1511225122 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f321415a780 0x7f32141f05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.994+0000 7f321a2a9700 1 -- 172.21.15.22:0/1511225122 >> 172.21.15.22:0/1511225122 conn(0x7f321406b110 msgr2=0x7f32140af700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.994+0000 7f321a2a9700 1 -- 172.21.15.22:0/1511225122 shutdown_connections 2023-07-06T01:21:39.999 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:39.994+0000 7f321a2a9700 1 -- 172.21.15.22:0/1511225122 wait complete. 2023-07-06T01:21:40.008 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-07-06T01:19:17.673760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-07-06T01:19:18.054792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T01:20:22.043370+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T01:20:23.225298+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T01:21:40.009 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T01:21:40.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.135+0000 7f8407b27700 1 Processor -- start 2023-07-06T01:21:40.137 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.135+0000 7f8407b27700 1 -- start start 2023-07-06T01:21:40.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.136+0000 7f8407b27700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f840006a710 0x7f84000ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.136+0000 7f8407b27700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84000ca2b0 con 0x7f840015cb50 2023-07-06T01:21:40.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.136+0000 7f8407b27700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84000cfc90 con 0x7f84000cf090 2023-07-06T01:21:40.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.136+0000 7f8407b27700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84000af440 con 0x7f840006a710 2023-07-06T01:21:40.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.136+0000 7f84058c3700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f840015cb50 0x7f840006a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38628/0 (socket says 172.21.15.22:38628) 2023-07-06T01:21:40.139 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.136+0000 7f84058c3700 1 -- 172.21.15.22:0/4256445909 learned_addr learned my addr 172.21.15.22:0/4256445909 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:40.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.137+0000 7f84050c2700 1 --2- 172.21.15.22:0/4256445909 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f840006a710 0x7f84000ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.137+0000 7f84048c1700 1 -- 172.21.15.22:0/4256445909 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390862847 0 0) 0x7f84000ca2b0 con 0x7f840015cb50 2023-07-06T01:21:40.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.137+0000 7f84048c1700 1 -- 172.21.15.22:0/4256445909 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83ec0035b0 con 0x7f840015cb50 2023-07-06T01:21:40.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.137+0000 7f84048c1700 1 -- 172.21.15.22:0/4256445909 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2820565432 0 0) 0x7f84000cfc90 con 0x7f84000cf090 2023-07-06T01:21:40.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.137+0000 7f84048c1700 1 -- 172.21.15.22:0/4256445909 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84000ca2b0 con 0x7f84000cf090 2023-07-06T01:21:40.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.137+0000 7f84050c2700 1 -- 172.21.15.22:0/4256445909 >> v1:172.21.15.103:6789/0 conn(0x7f84000cf090 legacy=0x7f8400160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.137+0000 7f84050c2700 1 -- 172.21.15.22:0/4256445909 >> v1:172.21.15.22:6789/0 conn(0x7f840015cb50 legacy=0x7f840006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.140 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.137+0000 7f84050c2700 1 -- 172.21.15.22:0/4256445909 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84000cff30 con 0x7f840006a710 2023-07-06T01:21:40.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.137+0000 7f84050c2700 1 --2- 172.21.15.22:0/4256445909 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f840006a710 0x7f84000ceb50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009870 tx=0x7f83fc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e64066b803319781 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:40.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.137+0000 7f84048c1700 1 -- 172.21.15.22:0/4256445909 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc011830 con 0x7f840006a710 2023-07-06T01:21:40.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.137+0000 7f84048c1700 1 -- 172.21.15.22:0/4256445909 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc011990 con 0x7f840006a710 2023-07-06T01:21:40.141 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.137+0000 7f84048c1700 1 -- 172.21.15.22:0/4256445909 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc011d20 con 0x7f840006a710 2023-07-06T01:21:40.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.138+0000 7f8407b27700 1 -- 172.21.15.22:0/4256445909 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f840006a710 msgr2=0x7f84000ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.138+0000 7f8407b27700 1 --2- 172.21.15.22:0/4256445909 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f840006a710 0x7f84000ceb50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009870 tx=0x7f83fc009b80 comp rx=0 tx=0).stop 2023-07-06T01:21:40.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.138+0000 7f8407b27700 1 -- 172.21.15.22:0/4256445909 shutdown_connections 2023-07-06T01:21:40.143 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.138+0000 7f8407b27700 1 --2- 172.21.15.22:0/4256445909 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f840006a710 0x7f84000ceb50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.138+0000 7f8407b27700 1 -- 172.21.15.22:0/4256445909 >> 172.21.15.22:0/4256445909 conn(0x7f840006b110 msgr2=0x7f840006b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.138+0000 7f8407b27700 1 -- 172.21.15.22:0/4256445909 shutdown_connections 2023-07-06T01:21:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.138+0000 7f8407b27700 1 -- 172.21.15.22:0/4256445909 wait complete. 2023-07-06T01:21:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.139+0000 7f8407b27700 1 Processor -- start 2023-07-06T01:21:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.139+0000 7f8407b27700 1 -- start start 2023-07-06T01:21:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f8407b27700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f840006a710 0x7f8400152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f8407b27700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84000cf090 0x7f8400152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.144 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f8407b27700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f840015cb50 0x7f840015f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f8407b27700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84000c41a0 con 0x7f84000cf090 2023-07-06T01:21:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f8407b27700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84000c4d20 con 0x7f840015cb50 2023-07-06T01:21:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f8407b27700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84000c43f0 con 0x7f840006a710 2023-07-06T01:21:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f84050c2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84000cf090 0x7f8400152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f84050c2700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84000cf090 0x7f8400152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47892/0 (socket says 172.21.15.22:47892) 2023-07-06T01:21:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f84050c2700 1 -- 172.21.15.22:0/304075627 learned_addr learned my addr 172.21.15.22:0/304075627 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:40.145 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f84058c3700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f840006a710 0x7f8400152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f84060c4700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f840015cb50 0x7f840015f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f84050c2700 1 -- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f840006a710 msgr2=0x7f8400152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f84050c2700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f840006a710 0x7f8400152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f84050c2700 1 -- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f840015cb50 msgr2=0x7f840015f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f84050c2700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f840015cb50 0x7f840015f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f84050c2700 1 -- 172.21.15.22:0/304075627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc0094a0 con 0x7f84000cf090 2023-07-06T01:21:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.140+0000 7f84060c4700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f840015cb50 0x7f840015f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:40.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.141+0000 7f84050c2700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84000cf090 0x7f8400152920 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0090b0 tx=0x7f83fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.141+0000 7f83f67fc700 1 -- 172.21.15.22:0/304075627 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc01a760 con 0x7f84000cf090 2023-07-06T01:21:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.141+0000 7f83f67fc700 1 -- 172.21.15.22:0/304075627 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc01a8c0 con 0x7f84000cf090 2023-07-06T01:21:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.141+0000 7f83f67fc700 1 -- 172.21.15.22:0/304075627 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc01ac30 con 0x7f84000cf090 2023-07-06T01:21:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.141+0000 7f8407b27700 1 -- 172.21.15.22:0/304075627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84000c46d0 con 0x7f84000cf090 2023-07-06T01:21:40.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.141+0000 7f8407b27700 1 -- 172.21.15.22:0/304075627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84001536c0 con 0x7f84000cf090 2023-07-06T01:21:40.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.142+0000 7f8407b27700 1 -- 172.21.15.22:0/304075627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8400153880 con 0x7f84000cf090 2023-07-06T01:21:40.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.142+0000 7f83f67fc700 1 -- 172.21.15.22:0/304075627 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f83fc022070 con 0x7f84000cf090 2023-07-06T01:21:40.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.143+0000 7f83f67fc700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f83e406ace0 0x7f83e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.148 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.143+0000 7f83f67fc700 1 -- 172.21.15.22:0/304075627 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6147+0+0 (secure 0 0 0) 0x7f83fc08a110 con 0x7f84000cf090 2023-07-06T01:21:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.143+0000 7f84058c3700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f83e406ace0 0x7f83e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.143+0000 7f84058c3700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f83e406ace0 0x7f83e406d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f83f0007fd0 tx=0x7f83f0007e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:40.149 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.147+0000 7f83f67fc700 1 -- 172.21.15.22:0/304075627 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f83fc055210 con 0x7f84000cf090 2023-07-06T01:21:40.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.348+0000 7f8407b27700 1 -- 172.21.15.22:0/304075627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f84000c4e90 con 0x7f84000cf090 2023-07-06T01:21:40.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.348+0000 7f83f67fc700 1 -- 172.21.15.22:0/304075627 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7f83fc0150a0 con 0x7f84000cf090 2023-07-06T01:21:40.351 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 8 2023-07-06T01:21:40.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.350+0000 7f8407b27700 1 -- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f83e406ace0 msgr2=0x7f83e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.352 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.350+0000 7f8407b27700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f83e406ace0 0x7f83e406d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f83f0007fd0 tx=0x7f83f0007e00 comp rx=0 tx=0).stop 2023-07-06T01:21:40.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.350+0000 7f8407b27700 1 -- 172.21.15.22:0/304075627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84000cf090 msgr2=0x7f8400152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.350+0000 7f8407b27700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84000cf090 0x7f8400152920 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0090b0 tx=0x7f83fc004750 comp rx=0 tx=0).stop 2023-07-06T01:21:40.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.351+0000 7f8407b27700 1 -- 172.21.15.22:0/304075627 shutdown_connections 2023-07-06T01:21:40.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.351+0000 7f8407b27700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f83e406ace0 0x7f83e406d190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.351+0000 7f8407b27700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f840015cb50 0x7f840015f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.351+0000 7f8407b27700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f840006a710 0x7f8400152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.353 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.351+0000 7f8407b27700 1 --2- 172.21.15.22:0/304075627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84000cf090 0x7f8400152920 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.351+0000 7f8407b27700 1 -- 172.21.15.22:0/304075627 >> 172.21.15.22:0/304075627 conn(0x7f840006b110 msgr2=0x7f84000b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:40.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.351+0000 7f8407b27700 1 -- 172.21.15.22:0/304075627 shutdown_connections 2023-07-06T01:21:40.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.351+0000 7f8407b27700 1 -- 172.21.15.22:0/304075627 wait complete. 2023-07-06T01:21:40.365 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T01:21:40.492 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.491+0000 7f138d53e700 1 Processor -- start 2023-07-06T01:21:40.493 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.491+0000 7f138d53e700 1 -- start start 2023-07-06T01:21:40.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f138d53e700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f13880b36f0 0x7f13880cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f138d53e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f13880cf5a0 0x7f13880d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f138d53e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13880b3b70 0x7f13880d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.495 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f138d53e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138815cff0 con 0x7f13880cf5a0 2023-07-06T01:21:40.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f138d53e700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13880cebd0 con 0x7f13880b36f0 2023-07-06T01:21:40.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f138d53e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13880cdfc0 con 0x7f13880b3b70 2023-07-06T01:21:40.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f13867fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f13880cf5a0 0x7f13880d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f13867fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f13880cf5a0 0x7f13880d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47898/0 (socket says 172.21.15.22:47898) 2023-07-06T01:21:40.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f13867fc700 1 -- 172.21.15.22:0/2124256538 learned_addr learned my addr 172.21.15.22:0/2124256538 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:40.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f13877fe700 1 --2- 172.21.15.22:0/2124256538 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13880b3b70 0x7f13880d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.496 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f1386ffd700 1 --2- 172.21.15.22:0/2124256538 >> v2:172.21.15.103:3300/0 conn(0x7f13880b36f0 0x7f13880cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f13867fc700 1 -- 172.21.15.22:0/2124256538 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13880b3b70 msgr2=0x7f13880d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:40.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f13867fc700 1 --2- 172.21.15.22:0/2124256538 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13880b3b70 0x7f13880d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f13867fc700 1 -- 172.21.15.22:0/2124256538 >> v2:172.21.15.103:3300/0 conn(0x7f13880b36f0 msgr2=0x7f13880cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:40.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f13867fc700 1 --2- 172.21.15.22:0/2124256538 >> v2:172.21.15.103:3300/0 conn(0x7f13880b36f0 0x7f13880cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f13867fc700 1 -- 172.21.15.22:0/2124256538 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880ce2a0 con 0x7f13880cf5a0 2023-07-06T01:21:40.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f1386ffd700 1 --2- 172.21.15.22:0/2124256538 >> v2:172.21.15.103:3300/0 conn(0x7f13880b36f0 0x7f13880cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:40.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f13867fc700 1 --2- 172.21.15.22:0/2124256538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13880cf5a0 0x7f13880d0a90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f138808b8d0 tx=0x7f1378009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4730cffe8ec3cd2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:40.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.492+0000 7f1385ffb700 1 -- 172.21.15.22:0/2124256538 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378011830 con 0x7f13880cf5a0 2023-07-06T01:21:40.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.493+0000 7f1385ffb700 1 -- 172.21.15.22:0/2124256538 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378011990 con 0x7f13880cf5a0 2023-07-06T01:21:40.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.493+0000 7f1385ffb700 1 -- 172.21.15.22:0/2124256538 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378011d20 con 0x7f13880cf5a0 2023-07-06T01:21:40.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.493+0000 7f138d53e700 1 -- 172.21.15.22:0/2124256538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13880cf5a0 msgr2=0x7f13880d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.493+0000 7f138d53e700 1 --2- 172.21.15.22:0/2124256538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13880cf5a0 0x7f13880d0a90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f138808b8d0 tx=0x7f1378009bd0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.493+0000 7f138d53e700 1 -- 172.21.15.22:0/2124256538 shutdown_connections 2023-07-06T01:21:40.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.493+0000 7f138d53e700 1 --2- 172.21.15.22:0/2124256538 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13880b3b70 0x7f13880d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.493+0000 7f138d53e700 1 --2- 172.21.15.22:0/2124256538 >> v2:172.21.15.103:3300/0 conn(0x7f13880b36f0 0x7f13880cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.493+0000 7f138d53e700 1 --2- 172.21.15.22:0/2124256538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13880cf5a0 0x7f13880d0a90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.493+0000 7f138d53e700 1 -- 172.21.15.22:0/2124256538 >> 172.21.15.22:0/2124256538 conn(0x7f138809c2e0 msgr2=0x7f13880d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:40.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.493+0000 7f138d53e700 1 -- 172.21.15.22:0/2124256538 shutdown_connections 2023-07-06T01:21:40.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.493+0000 7f138d53e700 1 -- 172.21.15.22:0/2124256538 wait complete. 2023-07-06T01:21:40.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.494+0000 7f138d53e700 1 Processor -- start 2023-07-06T01:21:40.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.495+0000 7f138d53e700 1 -- start start 2023-07-06T01:21:40.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.495+0000 7f138d53e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13880b36f0 0x7f13881f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.495+0000 7f138d53e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13880b3b70 0x7f13881f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.495+0000 7f138d53e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13880cf5a0 0x7f13880b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.495+0000 7f138d53e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13881f2d60 con 0x7f13880b3b70 2023-07-06T01:21:40.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.495+0000 7f138d53e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13881f6310 con 0x7f13880b36f0 2023-07-06T01:21:40.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.495+0000 7f138d53e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13881f3150 con 0x7f13880cf5a0 2023-07-06T01:21:40.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.495+0000 7f13867fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13880b3b70 0x7f13881f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.495+0000 7f13867fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13880b3b70 0x7f13881f2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47908/0 (socket says 172.21.15.22:47908) 2023-07-06T01:21:40.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.495+0000 7f13867fc700 1 -- 172.21.15.22:0/3239988694 learned_addr learned my addr 172.21.15.22:0/3239988694 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:40.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.496+0000 7f13877fe700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13880cf5a0 0x7f13880b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.496+0000 7f1386ffd700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13880b36f0 0x7f13881f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.496+0000 7f13867fc700 1 -- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13880cf5a0 msgr2=0x7f13880b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.496+0000 7f13867fc700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13880cf5a0 0x7f13880b7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.496+0000 7f13867fc700 1 -- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13880b36f0 msgr2=0x7f13881f53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.496+0000 7f13867fc700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13880b36f0 0x7f13881f53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.496+0000 7f13867fc700 1 -- 172.21.15.22:0/3239988694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1378009550 con 0x7f13880b3b70 2023-07-06T01:21:40.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.496+0000 7f13867fc700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13880b3b70 0x7f13881f2820 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1378000c00 tx=0x7f1378004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:40.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.496+0000 7f136f7fe700 1 -- 172.21.15.22:0/3239988694 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378004240 con 0x7f13880b3b70 2023-07-06T01:21:40.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.496+0000 7f136f7fe700 1 -- 172.21.15.22:0/3239988694 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137801f070 con 0x7f13880b3b70 2023-07-06T01:21:40.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.496+0000 7f138d53e700 1 -- 172.21.15.22:0/3239988694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13881f5a70 con 0x7f13880b3b70 2023-07-06T01:21:40.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.496+0000 7f136f7fe700 1 -- 172.21.15.22:0/3239988694 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13780036a0 con 0x7f13880b3b70 2023-07-06T01:21:40.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.496+0000 7f138d53e700 1 -- 172.21.15.22:0/3239988694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13881f6030 con 0x7f13880b3b70 2023-07-06T01:21:40.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.497+0000 7f138d53e700 1 -- 172.21.15.22:0/3239988694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13880b5010 con 0x7f13880b3b70 2023-07-06T01:21:40.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.497+0000 7f136f7fe700 1 -- 172.21.15.22:0/3239988694 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1378021030 con 0x7f13880b3b70 2023-07-06T01:21:40.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.498+0000 7f136f7fe700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f137406ace0 0x7f137406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.498+0000 7f136f7fe700 1 -- 172.21.15.22:0/3239988694 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6147+0+0 (secure 0 0 0) 0x7f137808a6d0 con 0x7f13880b3b70 2023-07-06T01:21:40.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.498+0000 7f1386ffd700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f137406ace0 0x7f137406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.499+0000 7f1386ffd700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f137406ace0 0x7f137406d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f137c0045f0 tx=0x7f137c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:40.506 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.502+0000 7f136f7fe700 1 -- 172.21.15.22:0/3239988694 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f13780558d0 con 0x7f13880b3b70 2023-07-06T01:21:40.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.698+0000 7f138d53e700 1 -- 172.21.15.22:0/3239988694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13880744b0 con 0x7f13880b3b70 2023-07-06T01:21:40.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.699+0000 7f136f7fe700 1 -- 172.21.15.22:0/3239988694 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1378088e60 con 0x7f13880b3b70 2023-07-06T01:21:40.701 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2023-07-06T01:21:40.702 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.701+0000 7f138d53e700 1 -- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f137406ace0 msgr2=0x7f137406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.701+0000 7f138d53e700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f137406ace0 0x7f137406d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f137c0045f0 tx=0x7f137c008040 comp rx=0 tx=0).stop 2023-07-06T01:21:40.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.701+0000 7f138d53e700 1 -- 172.21.15.22:0/3239988694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13880b3b70 msgr2=0x7f13881f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.701+0000 7f138d53e700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13880b3b70 0x7f13881f2820 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1378000c00 tx=0x7f1378004750 comp rx=0 tx=0).stop 2023-07-06T01:21:40.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.701+0000 7f138d53e700 1 -- 172.21.15.22:0/3239988694 shutdown_connections 2023-07-06T01:21:40.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.701+0000 7f138d53e700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13880cf5a0 0x7f13880b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.701+0000 7f138d53e700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f137406ace0 0x7f137406d190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.703 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.701+0000 7f138d53e700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13880b36f0 0x7f13881f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.701+0000 7f138d53e700 1 --2- 172.21.15.22:0/3239988694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13880b3b70 0x7f13881f2820 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.701+0000 7f138d53e700 1 -- 172.21.15.22:0/3239988694 >> 172.21.15.22:0/3239988694 conn(0x7f138809c2e0 msgr2=0x7f13880d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:40.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.701+0000 7f138d53e700 1 -- 172.21.15.22:0/3239988694 shutdown_connections 2023-07-06T01:21:40.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.701+0000 7f138d53e700 1 -- 172.21.15.22:0/3239988694 wait complete. 2023-07-06T01:21:40.714 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T01:21:40.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.841+0000 7f495a844700 1 Processor -- start 2023-07-06T01:21:40.843 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.842+0000 7f495a844700 1 -- start start 2023-07-06T01:21:40.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.842+0000 7f495a844700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f495415df70 0x7f49540cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.842+0000 7f495a844700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f49540d0130 0x7f49540ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.842+0000 7f495a844700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49540ce550 con 0x7f4954159870 2023-07-06T01:21:40.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.842+0000 7f495a844700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49540d05b0 con 0x7f49540d0130 2023-07-06T01:21:40.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.842+0000 7f495a844700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49540cdb60 con 0x7f495415df70 2023-07-06T01:21:40.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.842+0000 7f4958de1700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f49540d0130 0x7f49540ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.842+0000 7f4953fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f4954159870 0x7f49540715f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38656/0 (socket says 172.21.15.22:38656) 2023-07-06T01:21:40.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.842+0000 7f4953fff700 1 -- 172.21.15.22:0/2036191974 learned_addr learned my addr 172.21.15.22:0/2036191974 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:40.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.843+0000 7f4958de1700 1 -- 172.21.15.22:0/2036191974 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f495415df70 msgr2=0x7f49540cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:40.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.843+0000 7f49537fe700 1 --2- 172.21.15.22:0/2036191974 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f495415df70 0x7f49540cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.843+0000 7f4958de1700 1 --2- 172.21.15.22:0/2036191974 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f495415df70 0x7f49540cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.843+0000 7f4958de1700 1 -- 172.21.15.22:0/2036191974 >> v1:172.21.15.22:6789/0 conn(0x7f4954159870 legacy=0x7f49540715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.843+0000 7f4958de1700 1 -- 172.21.15.22:0/2036191974 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495415d490 con 0x7f49540d0130 2023-07-06T01:21:40.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.843+0000 7f4952ffd700 1 -- 172.21.15.22:0/2036191974 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845338494 0 0) 0x7f49540ce550 con 0x7f4954159870 2023-07-06T01:21:40.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.843+0000 7f49537fe700 1 --2- 172.21.15.22:0/2036191974 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f495415df70 0x7f49540cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:40.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.843+0000 7f4958de1700 1 --2- 172.21.15.22:0/2036191974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49540d0130 0x7f49540ca360 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f494c009d90 tx=0x7f494c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30980c405a39414a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:40.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.843+0000 7f4952ffd700 1 -- 172.21.15.22:0/2036191974 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c00e070 con 0x7f49540d0130 2023-07-06T01:21:40.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.843+0000 7f4952ffd700 1 -- 172.21.15.22:0/2036191974 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c00bbd0 con 0x7f49540d0130 2023-07-06T01:21:40.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.843+0000 7f4952ffd700 1 -- 172.21.15.22:0/2036191974 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c012430 con 0x7f49540d0130 2023-07-06T01:21:40.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.844+0000 7f495a844700 1 -- 172.21.15.22:0/2036191974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49540d0130 msgr2=0x7f49540ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.844+0000 7f495a844700 1 --2- 172.21.15.22:0/2036191974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49540d0130 0x7f49540ca360 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f494c009d90 tx=0x7f494c00b330 comp rx=0 tx=0).stop 2023-07-06T01:21:40.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.844+0000 7f495a844700 1 -- 172.21.15.22:0/2036191974 shutdown_connections 2023-07-06T01:21:40.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.844+0000 7f495a844700 1 --2- 172.21.15.22:0/2036191974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49540d0130 0x7f49540ca360 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.844+0000 7f495a844700 1 --2- 172.21.15.22:0/2036191974 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f495415df70 0x7f49540cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.844+0000 7f495a844700 1 -- 172.21.15.22:0/2036191974 >> 172.21.15.22:0/2036191974 conn(0x7f495406b110 msgr2=0x7f495406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:40.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.844+0000 7f495a844700 1 -- 172.21.15.22:0/2036191974 shutdown_connections 2023-07-06T01:21:40.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.844+0000 7f495a844700 1 -- 172.21.15.22:0/2036191974 wait complete. 2023-07-06T01:21:40.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.845+0000 7f495a844700 1 Processor -- start 2023-07-06T01:21:40.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.845+0000 7f495a844700 1 -- start start 2023-07-06T01:21:40.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f495a844700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49540d0130 0x7f49540aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f495a844700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4954159870 0x7f495406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f495a844700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f495415df70 0x7f49541f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f495a844700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49541f3310 con 0x7f49540d0130 2023-07-06T01:21:40.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f495a844700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49540aea60 con 0x7f495415df70 2023-07-06T01:21:40.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f495a844700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49540ae470 con 0x7f4954159870 2023-07-06T01:21:40.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f4953fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49540d0130 0x7f49540aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f4953fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49540d0130 0x7f49540aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47918/0 (socket says 172.21.15.22:47918) 2023-07-06T01:21:40.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f4953fff700 1 -- 172.21.15.22:0/19665591 learned_addr learned my addr 172.21.15.22:0/19665591 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:40.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f49537fe700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4954159870 0x7f495406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f4958de1700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f495415df70 0x7f49541f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f4953fff700 1 -- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4954159870 msgr2=0x7f495406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f4953fff700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4954159870 0x7f495406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f4953fff700 1 -- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f495415df70 msgr2=0x7f49541f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:40.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f4953fff700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f495415df70 0x7f49541f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:40.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f4953fff700 1 -- 172.21.15.22:0/19665591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c009a70 con 0x7f49540d0130 2023-07-06T01:21:40.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.846+0000 7f4953fff700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49540d0130 0x7f49540aeef0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f494400c750 tx=0x7f494400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:40.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.847+0000 7f4950ff9700 1 -- 172.21.15.22:0/19665591 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4944004c80 con 0x7f49540d0130 2023-07-06T01:21:40.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.847+0000 7f4950ff9700 1 -- 172.21.15.22:0/19665591 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4944004e00 con 0x7f49540d0130 2023-07-06T01:21:40.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.847+0000 7f495a844700 1 -- 172.21.15.22:0/19665591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49541f2d50 con 0x7f49540d0130 2023-07-06T01:21:40.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.847+0000 7f4950ff9700 1 -- 172.21.15.22:0/19665591 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4944007310 con 0x7f49540d0130 2023-07-06T01:21:40.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.847+0000 7f495a844700 1 -- 172.21.15.22:0/19665591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49540af430 con 0x7f49540d0130 2023-07-06T01:21:40.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.848+0000 7f495a844700 1 -- 172.21.15.22:0/19665591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4954066910 con 0x7f49540d0130 2023-07-06T01:21:40.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.848+0000 7f4950ff9700 1 -- 172.21.15.22:0/19665591 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4944007470 con 0x7f49540d0130 2023-07-06T01:21:40.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.848+0000 7f4950ff9700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f493406ace0 0x7f493406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:40.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.849+0000 7f4950ff9700 1 -- 172.21.15.22:0/19665591 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6147+0+0 (secure 0 0 0) 0x7f4944089110 con 0x7f49540d0130 2023-07-06T01:21:40.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.849+0000 7f49537fe700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f493406ace0 0x7f493406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:40.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.849+0000 7f49537fe700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f493406ace0 0x7f493406d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f49540af6b0 tx=0x7f493c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:40.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:40.853+0000 7f4950ff9700 1 -- 172.21.15.22:0/19665591 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4944054160 con 0x7f49540d0130 2023-07-06T01:21:41.053 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.051+0000 7f495a844700 1 -- 172.21.15.22:0/19665591 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49540aed40 con 0x7f49540d0130 2023-07-06T01:21:41.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.052+0000 7f4950ff9700 1 -- 172.21.15.22:0/19665591 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4944057780 con 0x7f49540d0130 2023-07-06T01:21:41.054 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2023-07-06T01:21:41.056 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.054+0000 7f495a844700 1 -- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f493406ace0 msgr2=0x7f493406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.054+0000 7f495a844700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f493406ace0 0x7f493406d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f49540af6b0 tx=0x7f493c008040 comp rx=0 tx=0).stop 2023-07-06T01:21:41.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.054+0000 7f495a844700 1 -- 172.21.15.22:0/19665591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49540d0130 msgr2=0x7f49540aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.054+0000 7f495a844700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49540d0130 0x7f49540aeef0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f494400c750 tx=0x7f494400ca60 comp rx=0 tx=0).stop 2023-07-06T01:21:41.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.054+0000 7f495a844700 1 -- 172.21.15.22:0/19665591 shutdown_connections 2023-07-06T01:21:41.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.054+0000 7f495a844700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f493406ace0 0x7f493406d190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.054+0000 7f495a844700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f495415df70 0x7f49541f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.054+0000 7f495a844700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49540d0130 0x7f49540aeef0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.054+0000 7f495a844700 1 --2- 172.21.15.22:0/19665591 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4954159870 0x7f495406b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.054+0000 7f495a844700 1 -- 172.21.15.22:0/19665591 >> 172.21.15.22:0/19665591 conn(0x7f495406b110 msgr2=0x7f49540af700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:41.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.054+0000 7f495a844700 1 -- 172.21.15.22:0/19665591 shutdown_connections 2023-07-06T01:21:41.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.054+0000 7f495a844700 1 -- 172.21.15.22:0/19665591 wait complete. 2023-07-06T01:21:41.068 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T01:21:41.197 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.196+0000 7f29ff458700 1 Processor -- start 2023-07-06T01:21:41.198 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.196+0000 7f29ff458700 1 -- start start 2023-07-06T01:21:41.199 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29ff458700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f29f815c940 0x7f29f80d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29ff458700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29f80d11a0 0x7f29f80afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29ff458700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f815cd10 con 0x7f29f80cd510 2023-07-06T01:21:41.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29ff458700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f809c8a0 con 0x7f29f815c940 2023-07-06T01:21:41.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29ff458700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f80ce710 con 0x7f29f80d11a0 2023-07-06T01:21:41.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29fd1f4700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f29f815c940 0x7f29f80d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.200 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29fd1f4700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f29f815c940 0x7f29f80d2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47926/0 (socket says 172.21.15.22:47926) 2023-07-06T01:21:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29fd1f4700 1 -- 172.21.15.22:0/1362412183 learned_addr learned my addr 172.21.15.22:0/1362412183 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29fd9f5700 1 --2- 172.21.15.22:0/1362412183 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29f80d11a0 0x7f29f80afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29fd1f4700 1 -- 172.21.15.22:0/1362412183 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29f80d11a0 msgr2=0x7f29f80afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29fd1f4700 1 --2- 172.21.15.22:0/1362412183 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29f80d11a0 0x7f29f80afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29fd1f4700 1 -- 172.21.15.22:0/1362412183 >> v1:172.21.15.103:6789/0 conn(0x7f29f80cd510 legacy=0x7f29f80d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29fd1f4700 1 -- 172.21.15.22:0/1362412183 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f80ce9f0 con 0x7f29f815c940 2023-07-06T01:21:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29fd1f4700 1 --2- 172.21.15.22:0/1362412183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29f815c940 0x7f29f80d2a10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f29f40098f0 tx=0x7f29f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9e276067e083b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29effff700 1 -- 172.21.15.22:0/1362412183 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4011830 con 0x7f29f815c940 2023-07-06T01:21:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29effff700 1 -- 172.21.15.22:0/1362412183 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f4011990 con 0x7f29f815c940 2023-07-06T01:21:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.197+0000 7f29effff700 1 -- 172.21.15.22:0/1362412183 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4011d20 con 0x7f29f815c940 2023-07-06T01:21:41.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.198+0000 7f29ff458700 1 -- 172.21.15.22:0/1362412183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29f815c940 msgr2=0x7f29f80d2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.198+0000 7f29ff458700 1 --2- 172.21.15.22:0/1362412183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29f815c940 0x7f29f80d2a10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f29f40098f0 tx=0x7f29f4009c00 comp rx=0 tx=0).stop 2023-07-06T01:21:41.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.198+0000 7f29ff458700 1 -- 172.21.15.22:0/1362412183 shutdown_connections 2023-07-06T01:21:41.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.198+0000 7f29ff458700 1 --2- 172.21.15.22:0/1362412183 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29f80d11a0 0x7f29f80afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.198+0000 7f29ff458700 1 --2- 172.21.15.22:0/1362412183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29f815c940 0x7f29f80d2a10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.198+0000 7f29ff458700 1 -- 172.21.15.22:0/1362412183 >> 172.21.15.22:0/1362412183 conn(0x7f29f809c2e0 msgr2=0x7f29f80d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:41.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.198+0000 7f29ff458700 1 -- 172.21.15.22:0/1362412183 shutdown_connections 2023-07-06T01:21:41.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.198+0000 7f29ff458700 1 -- 172.21.15.22:0/1362412183 wait complete. 2023-07-06T01:21:41.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.199+0000 7f29ff458700 1 Processor -- start 2023-07-06T01:21:41.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.199+0000 7f29ff458700 1 -- start start 2023-07-06T01:21:41.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.200+0000 7f29ff458700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29f80cd510 0x7f29f814d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.200+0000 7f29ff458700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29f80d11a0 0x7f29f815cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.200+0000 7f29ff458700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29f815c940 0x7f29f815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.200+0000 7f29ff458700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f80ca620 con 0x7f29f815c940 2023-07-06T01:21:41.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.200+0000 7f29ff458700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f80b85a0 con 0x7f29f80d11a0 2023-07-06T01:21:41.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.200+0000 7f29ff458700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f80ca870 con 0x7f29f80cd510 2023-07-06T01:21:41.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.200+0000 7f29fd9f5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29f815c940 0x7f29f815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.200+0000 7f29fd9f5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29f815c940 0x7f29f815d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47936/0 (socket says 172.21.15.22:47936) 2023-07-06T01:21:41.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.200+0000 7f29fd9f5700 1 -- 172.21.15.22:0/2499829319 learned_addr learned my addr 172.21.15.22:0/2499829319 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:41.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.200+0000 7f29fd1f4700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29f80cd510 0x7f29f814d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.200+0000 7f29fc9f3700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29f80d11a0 0x7f29f815cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.201+0000 7f29fd9f5700 1 -- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29f80cd510 msgr2=0x7f29f814d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.201+0000 7f29fd9f5700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29f80cd510 0x7f29f814d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.201+0000 7f29fd9f5700 1 -- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29f80d11a0 msgr2=0x7f29f815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.201+0000 7f29fd9f5700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29f80d11a0 0x7f29f815cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.201+0000 7f29fd9f5700 1 -- 172.21.15.22:0/2499829319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f4009550 con 0x7f29f815c940 2023-07-06T01:21:41.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.201+0000 7f29fc9f3700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29f80d11a0 0x7f29f815cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:41.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.201+0000 7f29fd1f4700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29f80cd510 0x7f29f814d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:41.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.201+0000 7f29fd9f5700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29f815c940 0x7f29f815d3e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f29f000ef90 tx=0x7f29f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:41.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.201+0000 7f29edffb700 1 -- 172.21.15.22:0/2499829319 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f0009ca0 con 0x7f29f815c940 2023-07-06T01:21:41.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.201+0000 7f29edffb700 1 -- 172.21.15.22:0/2499829319 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f000f040 con 0x7f29f815c940 2023-07-06T01:21:41.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.201+0000 7f29edffb700 1 -- 172.21.15.22:0/2499829319 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f0014480 con 0x7f29f815c940 2023-07-06T01:21:41.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.201+0000 7f29ff458700 1 -- 172.21.15.22:0/2499829319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f80cab50 con 0x7f29f815c940 2023-07-06T01:21:41.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.201+0000 7f29ff458700 1 -- 172.21.15.22:0/2499829319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f814da40 con 0x7f29f815c940 2023-07-06T01:21:41.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.202+0000 7f29ff458700 1 -- 172.21.15.22:0/2499829319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f80cb070 con 0x7f29f815c940 2023-07-06T01:21:41.209 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.202+0000 7f29edffb700 1 -- 172.21.15.22:0/2499829319 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f29f00145e0 con 0x7f29f815c940 2023-07-06T01:21:41.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.203+0000 7f29edffb700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f29dc06ace0 0x7f29dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.203+0000 7f29edffb700 1 -- 172.21.15.22:0/2499829319 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f29f008a190 con 0x7f29f815c940 2023-07-06T01:21:41.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.203+0000 7f29fd1f4700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f29dc06ace0 0x7f29dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.204+0000 7f29fd1f4700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f29dc06ace0 0x7f29dc06d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f29f4014040 tx=0x7f29f4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:41.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.207+0000 7f29edffb700 1 -- 172.21.15.22:0/2499829319 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f29f0058c90 con 0x7f29f815c940 2023-07-06T01:21:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.407+0000 7f29ff458700 1 -- 172.21.15.22:0/2499829319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f29f80744b0 con 0x7f29f815c940 2023-07-06T01:21:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.407+0000 7f29edffb700 1 -- 172.21.15.22:0/2499829319 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7f29f0058820 con 0x7f29f815c940 2023-07-06T01:21:41.410 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2023-07-06T01:21:41.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.410+0000 7f29ff458700 1 -- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f29dc06ace0 msgr2=0x7f29dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.410+0000 7f29ff458700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f29dc06ace0 0x7f29dc06d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f29f4014040 tx=0x7f29f4008030 comp rx=0 tx=0).stop 2023-07-06T01:21:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.410+0000 7f29ff458700 1 -- 172.21.15.22:0/2499829319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29f815c940 msgr2=0x7f29f815d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.410+0000 7f29ff458700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29f815c940 0x7f29f815d3e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f29f000ef90 tx=0x7f29f000c3a0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.410+0000 7f29ff458700 1 -- 172.21.15.22:0/2499829319 shutdown_connections 2023-07-06T01:21:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.410+0000 7f29ff458700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29f815c940 0x7f29f815d3e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.410+0000 7f29ff458700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29f80cd510 0x7f29f814d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.410+0000 7f29ff458700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f29dc06ace0 0x7f29dc06d190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.410+0000 7f29ff458700 1 --2- 172.21.15.22:0/2499829319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29f80d11a0 0x7f29f815cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.410+0000 7f29ff458700 1 -- 172.21.15.22:0/2499829319 >> 172.21.15.22:0/2499829319 conn(0x7f29f809c2e0 msgr2=0x7f29f80d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.410+0000 7f29ff458700 1 -- 172.21.15.22:0/2499829319 shutdown_connections 2023-07-06T01:21:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.410+0000 7f29ff458700 1 -- 172.21.15.22:0/2499829319 wait complete. 2023-07-06T01:21:41.423 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:21:41.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.547+0000 7f6cd85ad700 1 Processor -- start 2023-07-06T01:21:41.549 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.547+0000 7f6cd85ad700 1 -- start start 2023-07-06T01:21:41.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.548+0000 7f6cd85ad700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cd015f100 0x7f6cd00d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.548+0000 7f6cd85ad700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd00cc4b0 con 0x7f6cd006a710 2023-07-06T01:21:41.551 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.548+0000 7f6cd85ad700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd015e3d0 con 0x7f6cd015a780 2023-07-06T01:21:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.548+0000 7f6cd85ad700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd015ec70 con 0x7f6cd015f100 2023-07-06T01:21:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.548+0000 7f6cd5b48700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6cd006a710 0x7f6cd00cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38684/0 (socket says 172.21.15.22:38684) 2023-07-06T01:21:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.548+0000 7f6cd5b48700 1 -- 172.21.15.22:0/4277257535 learned_addr learned my addr 172.21.15.22:0/4277257535 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.548+0000 7f6cd6b4a700 1 --2- 172.21.15.22:0/4277257535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cd015f100 0x7f6cd00d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367054190 0 0) 0x7f6ccc001110 con 0x7f6cd006a710 2023-07-06T01:21:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cbc0035b0 con 0x7f6cd006a710 2023-07-06T01:21:41.552 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3468173622 0 0) 0x7f6cbc0035b0 con 0x7f6cd006a710 2023-07-06T01:21:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cd015f100 msgr2=0x7f6cd00d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 --2- 172.21.15.22:0/4277257535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cd015f100 0x7f6cd00d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 >> v1:172.21.15.103:6789/0 conn(0x7f6cd015a780 legacy=0x7f6cd006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd00d0920 con 0x7f6cd006a710 2023-07-06T01:21:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f6ccc001800 con 0x7f6cd006a710 2023-07-06T01:21:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 >> v1:172.21.15.22:6789/0 conn(0x7f6cd006a710 legacy=0x7f6cd00cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 --2- 172.21.15.22:0/4277257535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cbc003a30 0x7f6cbc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.553 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 --2- 172.21.15.22:0/4277257535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cd015f100 0x7f6cbc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 --2- 172.21.15.22:0/4277257535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc004d90 0x7f6cbc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd015ec70 con 0x7f6cbc004d90 2023-07-06T01:21:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc00ba40 con 0x7f6cbc003a30 2023-07-06T01:21:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc00bc00 con 0x7f6cd015f100 2023-07-06T01:21:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd6b4a700 1 --2- 172.21.15.22:0/4277257535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc004d90 0x7f6cbc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd5b48700 1 --2- 172.21.15.22:0/4277257535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cd015f100 0x7f6cbc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.554 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd6349700 1 --2- 172.21.15.22:0/4277257535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cbc003a30 0x7f6cbc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd6b4a700 1 -- 172.21.15.22:0/4277257535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cd015f100 msgr2=0x7f6cbc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd6b4a700 1 --2- 172.21.15.22:0/4277257535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cd015f100 0x7f6cbc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd6b4a700 1 -- 172.21.15.22:0/4277257535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cbc003a30 msgr2=0x7f6cbc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd6b4a700 1 --2- 172.21.15.22:0/4277257535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cbc003a30 0x7f6cbc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd6b4a700 1 -- 172.21.15.22:0/4277257535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc0035b0 con 0x7f6cbc004d90 2023-07-06T01:21:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.549+0000 7f6cd6b4a700 1 --2- 172.21.15.22:0/4277257535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc004d90 0x7f6cbc005200 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00cb290 tx=0x7f6cbc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73c7dbe4bd8523f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.550+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc800eba0 con 0x7f6cbc004d90 2023-07-06T01:21:41.555 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.550+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc800f040 con 0x7f6cbc004d90 2023-07-06T01:21:41.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.550+0000 7f6cd5347700 1 -- 172.21.15.22:0/4277257535 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8014480 con 0x7f6cbc004d90 2023-07-06T01:21:41.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.550+0000 7f6cd85ad700 1 -- 172.21.15.22:0/4277257535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc004d90 msgr2=0x7f6cbc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.550+0000 7f6cd85ad700 1 --2- 172.21.15.22:0/4277257535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cbc004d90 0x7f6cbc005200 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00cb290 tx=0x7f6cbc005880 comp rx=0 tx=0).stop 2023-07-06T01:21:41.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.550+0000 7f6cd6b4a700 1 -- 172.21.15.22:0/4277257535 reap_dead start 2023-07-06T01:21:41.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.550+0000 7f6cd85ad700 1 -- 172.21.15.22:0/4277257535 shutdown_connections 2023-07-06T01:21:41.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.550+0000 7f6cd85ad700 1 -- 172.21.15.22:0/4277257535 >> 172.21.15.22:0/4277257535 conn(0x7f6cd006b110 msgr2=0x7f6cd006b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:41.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.550+0000 7f6cd85ad700 1 -- 172.21.15.22:0/4277257535 shutdown_connections 2023-07-06T01:21:41.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.550+0000 7f6cd85ad700 1 -- 172.21.15.22:0/4277257535 wait complete. 2023-07-06T01:21:41.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.552+0000 7f6cd85ad700 1 Processor -- start 2023-07-06T01:21:41.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.552+0000 7f6cd85ad700 1 -- start start 2023-07-06T01:21:41.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd85ad700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cbc004d90 0x7f6cd0071260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd85ad700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cd01540e0 0x7f6cd014a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd85ad700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd015cfc0 0x7f6cd015d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd85ad700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd015d970 con 0x7f6cd01540e0 2023-07-06T01:21:41.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd85ad700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00c6d30 con 0x7f6cd015cfc0 2023-07-06T01:21:41.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd85ad700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00c6f80 con 0x7f6cbc004d90 2023-07-06T01:21:41.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd5b48700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cd01540e0 0x7f6cd014a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd5b48700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cd01540e0 0x7f6cd014a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47954/0 (socket says 172.21.15.22:47954) 2023-07-06T01:21:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd5b48700 1 -- 172.21.15.22:0/3913546984 learned_addr learned my addr 172.21.15.22:0/3913546984 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd6349700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cbc004d90 0x7f6cd0071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd6b4a700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd015cfc0 0x7f6cd015d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd5b48700 1 -- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cbc004d90 msgr2=0x7f6cd0071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd5b48700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cbc004d90 0x7f6cd0071260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd5b48700 1 -- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd015cfc0 msgr2=0x7f6cd015d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd5b48700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd015cfc0 0x7f6cd015d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.561 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd5b48700 1 -- 172.21.15.22:0/3913546984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd00c7260 con 0x7f6cd01540e0 2023-07-06T01:21:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd6349700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cbc004d90 0x7f6cd0071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.553+0000 7f6cd5b48700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cd01540e0 0x7f6cd014a030 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0072c0 tx=0x7f6ccc007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.554+0000 7f6cc6ffd700 1 -- 172.21.15.22:0/3913546984 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0077a0 con 0x7f6cd01540e0 2023-07-06T01:21:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.554+0000 7f6cc6ffd700 1 -- 172.21.15.22:0/3913546984 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc00f870 con 0x7f6cd01540e0 2023-07-06T01:21:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.554+0000 7f6cd85ad700 1 -- 172.21.15.22:0/3913546984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd00c7540 con 0x7f6cd01540e0 2023-07-06T01:21:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.554+0000 7f6cc6ffd700 1 -- 172.21.15.22:0/3913546984 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc00fc20 con 0x7f6cd01540e0 2023-07-06T01:21:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.554+0000 7f6cd85ad700 1 -- 172.21.15.22:0/3913546984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd014a570 con 0x7f6cd01540e0 2023-07-06T01:21:41.562 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.554+0000 7f6cd85ad700 1 -- 172.21.15.22:0/3913546984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd00c7a60 con 0x7f6cd01540e0 2023-07-06T01:21:41.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.555+0000 7f6cc6ffd700 1 -- 172.21.15.22:0/3913546984 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ccc00fd80 con 0x7f6cd01540e0 2023-07-06T01:21:41.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.555+0000 7f6cc6ffd700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6cb406ace0 0x7f6cb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.556+0000 7f6cc6ffd700 1 -- 172.21.15.22:0/3913546984 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f6ccc089350 con 0x7f6cd01540e0 2023-07-06T01:21:41.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.556+0000 7f6cd6349700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6cb406ace0 0x7f6cb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.563 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.556+0000 7f6cd6349700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6cb406ace0 0x7f6cb406d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00c68f0 tx=0x7f6cc0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:41.564 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.560+0000 7f6cc6ffd700 1 -- 172.21.15.22:0/3913546984 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6cd00c7a60 con 0x7f6cd01540e0 2023-07-06T01:21:41.806 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.805+0000 7f6cd85ad700 1 -- 172.21.15.22:0/3913546984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cd00cc4b0 con 0x7f6cd01540e0 2023-07-06T01:21:41.807 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.805+0000 7f6cc6ffd700 1 -- 172.21.15.22:0/3913546984 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4672 (secure 0 0 0) 0x7f6cd00cc4b0 con 0x7f6cd01540e0 2023-07-06T01:21:41.807 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:21:41.807 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/219695916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":219695916},{"type":"v1","addr":"172.21.15.103:6836","nonce":219695916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3102932229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":3102932229},{"type":"v1","addr":"172.21.15.103:6837","nonce":3102932229}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3787686782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3787686782},{"type":"v1","addr":"172.21.15.22:6834","nonce":3787686782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-07-06T01:20:24.257974+0000","modified":"2023-07-06T01:21:37.327615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":22,"addr":"172.21.15.22:6835/612827809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":612827809},{"type":"v1","addr":"172.21.15.22:6835","nonce":612827809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T01:21:41.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.807+0000 7f6cd85ad700 1 -- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6cb406ace0 msgr2=0x7f6cb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.807+0000 7f6cd85ad700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6cb406ace0 0x7f6cb406d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00c68f0 tx=0x7f6cc0009040 comp rx=0 tx=0).stop 2023-07-06T01:21:41.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.807+0000 7f6cd85ad700 1 -- 172.21.15.22:0/3913546984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cd01540e0 msgr2=0x7f6cd014a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.807+0000 7f6cd85ad700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cd01540e0 0x7f6cd014a030 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0072c0 tx=0x7f6ccc007680 comp rx=0 tx=0).stop 2023-07-06T01:21:41.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.807+0000 7f6cd85ad700 1 -- 172.21.15.22:0/3913546984 shutdown_connections 2023-07-06T01:21:41.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.807+0000 7f6cd85ad700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6cb406ace0 0x7f6cb406d190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.807+0000 7f6cd85ad700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6cd015cfc0 0x7f6cd015d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.807+0000 7f6cd85ad700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6cbc004d90 0x7f6cd0071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.807+0000 7f6cd85ad700 1 --2- 172.21.15.22:0/3913546984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cd01540e0 0x7f6cd014a030 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.807+0000 7f6cd85ad700 1 -- 172.21.15.22:0/3913546984 >> 172.21.15.22:0/3913546984 conn(0x7f6cd006b110 msgr2=0x7f6cd00af720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:41.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.807+0000 7f6cd85ad700 1 -- 172.21.15.22:0/3913546984 shutdown_connections 2023-07-06T01:21:41.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.807+0000 7f6cd85ad700 1 -- 172.21.15.22:0/3913546984 wait complete. 2023-07-06T01:21:41.811 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2023-07-06T01:21:41.821 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:21:41.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.944+0000 7fa85632c700 1 Processor -- start 2023-07-06T01:21:41.946 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.944+0000 7fa85632c700 1 -- start start 2023-07-06T01:21:41.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.945+0000 7fa85632c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8500cc320 0x7fa85006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.945+0000 7fa85632c700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8500af670 con 0x7fa85006a710 2023-07-06T01:21:41.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.945+0000 7fa85632c700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85006d030 con 0x7fa8500af2a0 2023-07-06T01:21:41.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.945+0000 7fa85632c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8500cc780 con 0x7fa8500cc320 2023-07-06T01:21:41.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.945+0000 7fa84f7fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa85006a710 0x7fa8500cbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38706/0 (socket says 172.21.15.22:38706) 2023-07-06T01:21:41.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.945+0000 7fa84f7fe700 1 -- 172.21.15.22:0/2562038484 learned_addr learned my addr 172.21.15.22:0/2562038484 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:41.950 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa8548c9700 1 --2- 172.21.15.22:0/2562038484 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8500cc320 0x7fa85006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268570980 0 0) 0x7fa8500af670 con 0x7fa85006a710 2023-07-06T01:21:41.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa83c0035b0 con 0x7fa85006a710 2023-07-06T01:21:41.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2198335759 0 0) 0x7fa83c0035b0 con 0x7fa85006a710 2023-07-06T01:21:41.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8500cc320 msgr2=0x7fa85006caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:41.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 --2- 172.21.15.22:0/2562038484 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8500cc320 0x7fa85006caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 >> v1:172.21.15.103:6789/0 conn(0x7fa8500af2a0 legacy=0x7fa85006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8500cd560 con 0x7fa85006a710 2023-07-06T01:21:41.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7fa840001670 con 0x7fa85006a710 2023-07-06T01:21:41.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 >> v1:172.21.15.22:6789/0 conn(0x7fa85006a710 legacy=0x7fa8500cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 --2- 172.21.15.22:0/2562038484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa83c003a30 0x7fa83c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 --2- 172.21.15.22:0/2562038484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa83c004850 0x7fa83c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 --2- 172.21.15.22:0/2562038484 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8500cc320 0x7fa83c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8500cc780 con 0x7fa83c003a30 2023-07-06T01:21:41.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c00bac0 con 0x7fa83c004850 2023-07-06T01:21:41.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c00bc50 con 0x7fa8500cc320 2023-07-06T01:21:41.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84ffff700 1 --2- 172.21.15.22:0/2562038484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa83c003a30 0x7fa83c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84f7fe700 1 --2- 172.21.15.22:0/2562038484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa83c004850 0x7fa83c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa8548c9700 1 --2- 172.21.15.22:0/2562038484 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8500cc320 0x7fa83c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84ffff700 1 -- 172.21.15.22:0/2562038484 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8500cc320 msgr2=0x7fa83c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:41.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84ffff700 1 --2- 172.21.15.22:0/2562038484 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8500cc320 0x7fa83c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84ffff700 1 -- 172.21.15.22:0/2562038484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa83c004850 msgr2=0x7fa83c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:41.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84ffff700 1 --2- 172.21.15.22:0/2562038484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa83c004850 0x7fa83c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.946+0000 7fa84ffff700 1 -- 172.21.15.22:0/2562038484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c0035b0 con 0x7fa83c003a30 2023-07-06T01:21:41.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.947+0000 7fa84ffff700 1 --2- 172.21.15.22:0/2562038484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa83c003a30 0x7fa83c003e40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa8380017c0 tx=0x7fa83c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c685a263c7df1b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:41.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.947+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8380052d0 con 0x7fa83c003a30 2023-07-06T01:21:41.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.947+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa838005430 con 0x7fa83c003a30 2023-07-06T01:21:41.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.947+0000 7fa84effd700 1 -- 172.21.15.22:0/2562038484 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8380038c0 con 0x7fa83c003a30 2023-07-06T01:21:41.954 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.947+0000 7fa85632c700 1 -- 172.21.15.22:0/2562038484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa83c003a30 msgr2=0x7fa83c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.947+0000 7fa85632c700 1 --2- 172.21.15.22:0/2562038484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa83c003a30 0x7fa83c003e40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa8380017c0 tx=0x7fa83c005880 comp rx=0 tx=0).stop 2023-07-06T01:21:41.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.947+0000 7fa8548c9700 1 -- 172.21.15.22:0/2562038484 reap_dead start 2023-07-06T01:21:41.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.947+0000 7fa85632c700 1 -- 172.21.15.22:0/2562038484 shutdown_connections 2023-07-06T01:21:41.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.947+0000 7fa85632c700 1 -- 172.21.15.22:0/2562038484 >> 172.21.15.22:0/2562038484 conn(0x7fa8500b1890 msgr2=0x7fa8500b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:41.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.947+0000 7fa85632c700 1 -- 172.21.15.22:0/2562038484 shutdown_connections 2023-07-06T01:21:41.956 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.947+0000 7fa85632c700 1 -- 172.21.15.22:0/2562038484 wait complete. 2023-07-06T01:21:41.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.949+0000 7fa85632c700 1 Processor -- start 2023-07-06T01:21:41.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.949+0000 7fa85632c700 1 -- start start 2023-07-06T01:21:41.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.949+0000 7fa85632c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa83c003a30 0x7fa850152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa85632c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85015b590 0x7fa8500c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa85632c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8501f26a0 0x7fa8500ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.959 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa85632c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8500c9e70 con 0x7fa83c003a30 2023-07-06T01:21:41.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa85632c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85015dfe0 con 0x7fa85015b590 2023-07-06T01:21:41.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa85632c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85015e2c0 con 0x7fa8501f26a0 2023-07-06T01:21:41.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa83c003a30 0x7fa850152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84ffff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa83c003a30 0x7fa850152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47972/0 (socket says 172.21.15.22:47972) 2023-07-06T01:21:41.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84ffff700 1 -- 172.21.15.22:0/4106157355 learned_addr learned my addr 172.21.15.22:0/4106157355 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:41.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84f7fe700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85015b590 0x7fa8500c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.960 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa8548c9700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8501f26a0 0x7fa8500ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84ffff700 1 -- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8501f26a0 msgr2=0x7fa8500ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84ffff700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8501f26a0 0x7fa8500ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84ffff700 1 -- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85015b590 msgr2=0x7fa8500c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:41.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84ffff700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85015b590 0x7fa8500c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:41.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84ffff700 1 -- 172.21.15.22:0/4106157355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85015e5a0 con 0x7fa83c003a30 2023-07-06T01:21:41.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84f7fe700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85015b590 0x7fa8500c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:41.961 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84ffff700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa83c003a30 0x7fa850152160 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa838000c00 tx=0x7fa838001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:41.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84cff9700 1 -- 172.21.15.22:0/4106157355 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838007240 con 0x7fa83c003a30 2023-07-06T01:21:41.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84cff9700 1 -- 172.21.15.22:0/4106157355 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8380073a0 con 0x7fa83c003a30 2023-07-06T01:21:41.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa85632c700 1 -- 172.21.15.22:0/4106157355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85015e880 con 0x7fa83c003a30 2023-07-06T01:21:41.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa84cff9700 1 -- 172.21.15.22:0/4106157355 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838001290 con 0x7fa83c003a30 2023-07-06T01:21:41.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.950+0000 7fa85632c700 1 -- 172.21.15.22:0/4106157355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8501f2b10 con 0x7fa83c003a30 2023-07-06T01:21:41.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.951+0000 7fa85632c700 1 -- 172.21.15.22:0/4106157355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8500744b0 con 0x7fa83c003a30 2023-07-06T01:21:41.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.952+0000 7fa84cff9700 1 -- 172.21.15.22:0/4106157355 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa838003680 con 0x7fa83c003a30 2023-07-06T01:21:41.962 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.952+0000 7fa84cff9700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa83006ace0 0x7fa83006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:41.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.952+0000 7fa84cff9700 1 -- 172.21.15.22:0/4106157355 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fa838089f50 con 0x7fa83c003a30 2023-07-06T01:21:41.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.952+0000 7fa84f7fe700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa83006ace0 0x7fa83006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:41.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.953+0000 7fa84f7fe700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa83006ace0 0x7fa83006d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa8500c96c0 tx=0x7fa840009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:41.963 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:41.957+0000 7fa84cff9700 1 -- 172.21.15.22:0/4106157355 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa838055050 con 0x7fa83c003a30 2023-07-06T01:21:42.158 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.156+0000 7fa85632c700 1 -- 172.21.15.22:0/4106157355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8500712e0 con 0x7fa83c003a30 2023-07-06T01:21:42.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.158+0000 7fa84cff9700 1 -- 172.21.15.22:0/4106157355 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18339 (secure 0 0 0) 0x7fa838058670 con 0x7fa83c003a30 2023-07-06T01:21:42.160 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:21:42.160 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":38,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:21:41.176818+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:20:22.043370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T01:20:23.225298+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:21:42.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.160+0000 7fa85632c700 1 -- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa83006ace0 msgr2=0x7fa83006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:42.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.160+0000 7fa85632c700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa83006ace0 0x7fa83006d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa8500c96c0 tx=0x7fa840009040 comp rx=0 tx=0).stop 2023-07-06T01:21:42.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.160+0000 7fa85632c700 1 -- 172.21.15.22:0/4106157355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa83c003a30 msgr2=0x7fa850152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:42.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.160+0000 7fa85632c700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa83c003a30 0x7fa850152160 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa838000c00 tx=0x7fa838001120 comp rx=0 tx=0).stop 2023-07-06T01:21:42.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.160+0000 7fa85632c700 1 -- 172.21.15.22:0/4106157355 shutdown_connections 2023-07-06T01:21:42.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.160+0000 7fa85632c700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa8501f26a0 0x7fa8500ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.160+0000 7fa85632c700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa83c003a30 0x7fa850152160 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.160+0000 7fa85632c700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fa83006ace0 0x7fa83006d190 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.160+0000 7fa85632c700 1 --2- 172.21.15.22:0/4106157355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa85015b590 0x7fa8500c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.160+0000 7fa85632c700 1 -- 172.21.15.22:0/4106157355 >> 172.21.15.22:0/4106157355 conn(0x7fa8500b1890 msgr2=0x7fa8500b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:42.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.160+0000 7fa85632c700 1 -- 172.21.15.22:0/4106157355 shutdown_connections 2023-07-06T01:21:42.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.160+0000 7fa85632c700 1 -- 172.21.15.22:0/4106157355 wait complete. 2023-07-06T01:21:42.174 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T01:21:42.174 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:21:42.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.297+0000 7f70335ee700 1 Processor -- start 2023-07-06T01:21:42.299 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.298+0000 7f70335ee700 1 -- start start 2023-07-06T01:21:42.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.298+0000 7f70335ee700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f702c0af2a0 0x7f702c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.298+0000 7f70335ee700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f702c06a000 0x7f702c06a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.298+0000 7f70335ee700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f702c0cbc10 0x7f702c0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.298+0000 7f70335ee700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f702c06a9e0 con 0x7f702c0cbc10 2023-07-06T01:21:42.301 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.298+0000 7f70335ee700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f702c0af670 con 0x7f702c0af2a0 2023-07-06T01:21:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.298+0000 7f70335ee700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702c06d030 con 0x7f702c06a000 2023-07-06T01:21:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.298+0000 7f7031b8b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f702c0cbc10 0x7f702c0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.299+0000 7f7031b8b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f702c0cbc10 0x7f702c0cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47982/0 (socket says 172.21.15.22:47982) 2023-07-06T01:21:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.299+0000 7f7031b8b700 1 -- 172.21.15.22:0/1493714206 learned_addr learned my addr 172.21.15.22:0/1493714206 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.299+0000 7f7030b89700 1 --2- 172.21.15.22:0/1493714206 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f702c06a000 0x7f702c06a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.299+0000 7f703138a700 1 --2- 172.21.15.22:0/1493714206 >> v2:172.21.15.103:3300/0 conn(0x7f702c0af2a0 0x7f702c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.299+0000 7f7031b8b700 1 -- 172.21.15.22:0/1493714206 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f702c06a000 msgr2=0x7f702c06a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:42.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.299+0000 7f7031b8b700 1 --2- 172.21.15.22:0/1493714206 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f702c06a000 0x7f702c06a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.299+0000 7f7031b8b700 1 -- 172.21.15.22:0/1493714206 >> v2:172.21.15.103:3300/0 conn(0x7f702c0af2a0 msgr2=0x7f702c06caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:42.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.299+0000 7f7031b8b700 1 --2- 172.21.15.22:0/1493714206 >> v2:172.21.15.103:3300/0 conn(0x7f702c0af2a0 0x7f702c06caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.299+0000 7f7031b8b700 1 -- 172.21.15.22:0/1493714206 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c0cd560 con 0x7f702c0cbc10 2023-07-06T01:21:42.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.299+0000 7f7031b8b700 1 --2- 172.21.15.22:0/1493714206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f702c0cbc10 0x7f702c0cc090 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f702c067460 tx=0x7f702400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2b5622879cf82e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:42.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.299+0000 7f701ffff700 1 -- 172.21.15.22:0/1493714206 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702400e070 con 0x7f702c0cbc10 2023-07-06T01:21:42.303 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.299+0000 7f701ffff700 1 -- 172.21.15.22:0/1493714206 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702400b8d0 con 0x7f702c0cbc10 2023-07-06T01:21:42.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.299+0000 7f701ffff700 1 -- 172.21.15.22:0/1493714206 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702400bc80 con 0x7f702c0cbc10 2023-07-06T01:21:42.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.300+0000 7f70335ee700 1 -- 172.21.15.22:0/1493714206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f702c0cbc10 msgr2=0x7f702c0cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:42.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.300+0000 7f70335ee700 1 --2- 172.21.15.22:0/1493714206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f702c0cbc10 0x7f702c0cc090 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f702c067460 tx=0x7f702400b330 comp rx=0 tx=0).stop 2023-07-06T01:21:42.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.300+0000 7f70335ee700 1 -- 172.21.15.22:0/1493714206 shutdown_connections 2023-07-06T01:21:42.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.300+0000 7f70335ee700 1 --2- 172.21.15.22:0/1493714206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f702c0cbc10 0x7f702c0cc090 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.300+0000 7f70335ee700 1 --2- 172.21.15.22:0/1493714206 >> v2:172.21.15.103:3300/0 conn(0x7f702c0af2a0 0x7f702c06caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.300+0000 7f70335ee700 1 --2- 172.21.15.22:0/1493714206 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f702c06a000 0x7f702c06a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.300+0000 7f70335ee700 1 -- 172.21.15.22:0/1493714206 >> 172.21.15.22:0/1493714206 conn(0x7f702c0b1890 msgr2=0x7f702c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:42.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.300+0000 7f70335ee700 1 -- 172.21.15.22:0/1493714206 shutdown_connections 2023-07-06T01:21:42.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.300+0000 7f70335ee700 1 -- 172.21.15.22:0/1493714206 wait complete. 2023-07-06T01:21:42.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.301+0000 7f70335ee700 1 Processor -- start 2023-07-06T01:21:42.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.301+0000 7f70335ee700 1 -- start start 2023-07-06T01:21:42.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f70335ee700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f702c06a000 0x7f702c153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f70335ee700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f702c0af2a0 0x7f702c15a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f70335ee700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c0cbc10 0x7f702c0b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f70335ee700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c153c90 con 0x7f702c0af2a0 2023-07-06T01:21:42.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f70335ee700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c1f58e0 con 0x7f702c0cbc10 2023-07-06T01:21:42.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f70335ee700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702c1f5bc0 con 0x7f702c06a000 2023-07-06T01:21:42.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f7030b89700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f702c0af2a0 0x7f702c15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f7030b89700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f702c0af2a0 0x7f702c15a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47988/0 (socket says 172.21.15.22:47988) 2023-07-06T01:21:42.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f7030b89700 1 -- 172.21.15.22:0/1246119675 learned_addr learned my addr 172.21.15.22:0/1246119675 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:42.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f703138a700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f702c06a000 0x7f702c153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f7031b8b700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c0cbc10 0x7f702c0b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f7030b89700 1 -- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f702c06a000 msgr2=0x7f702c153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:42.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f7030b89700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f702c06a000 0x7f702c153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f7030b89700 1 -- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c0cbc10 msgr2=0x7f702c0b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:42.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f7030b89700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c0cbc10 0x7f702c0b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.309 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f7030b89700 1 -- 172.21.15.22:0/1246119675 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7024009b30 con 0x7f702c0af2a0 2023-07-06T01:21:42.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.302+0000 7f7030b89700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f702c0af2a0 0x7f702c15a800 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f702000e9b0 tx=0x7f702000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:42.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.303+0000 7f701dffb700 1 -- 172.21.15.22:0/1246119675 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020009710 con 0x7f702c0af2a0 2023-07-06T01:21:42.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.303+0000 7f701dffb700 1 -- 172.21.15.22:0/1246119675 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702000c850 con 0x7f702c0af2a0 2023-07-06T01:21:42.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.303+0000 7f70335ee700 1 -- 172.21.15.22:0/1246119675 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702c1f5ea0 con 0x7f702c0af2a0 2023-07-06T01:21:42.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.303+0000 7f701dffb700 1 -- 172.21.15.22:0/1246119675 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702000cbc0 con 0x7f702c0af2a0 2023-07-06T01:21:42.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.303+0000 7f70335ee700 1 -- 172.21.15.22:0/1246119675 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702c1f6460 con 0x7f702c0af2a0 2023-07-06T01:21:42.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.304+0000 7f70335ee700 1 -- 172.21.15.22:0/1246119675 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f702c0744b0 con 0x7f702c0af2a0 2023-07-06T01:21:42.310 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.304+0000 7f701dffb700 1 -- 172.21.15.22:0/1246119675 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f702000cd20 con 0x7f702c0af2a0 2023-07-06T01:21:42.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.305+0000 7f701dffb700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f701006ace0 0x7f701006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.311 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.305+0000 7f701dffb700 1 -- 172.21.15.22:0/1246119675 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f7020089670 con 0x7f702c0af2a0 2023-07-06T01:21:42.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.305+0000 7f703138a700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f701006ace0 0x7f701006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.306+0000 7f703138a700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f701006ace0 0x7f701006d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7028009a30 tx=0x7f7028008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:42.312 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.309+0000 7f701dffb700 1 -- 172.21.15.22:0/1246119675 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f70200580d0 con 0x7f702c0af2a0 2023-07-06T01:21:42.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.554+0000 7f70335ee700 1 -- 172.21.15.22:0/1246119675 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f702c15ca90 con 0x7f702c0af2a0 2023-07-06T01:21:42.556 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.555+0000 7f701dffb700 1 -- 172.21.15.22:0/1246119675 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4672 (secure 0 0 0) 0x7f702c15ca90 con 0x7f702c0af2a0 2023-07-06T01:21:42.557 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:21:42.557 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/219695916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":219695916},{"type":"v1","addr":"172.21.15.103:6836","nonce":219695916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3102932229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":3102932229},{"type":"v1","addr":"172.21.15.103:6837","nonce":3102932229}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3787686782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3787686782},{"type":"v1","addr":"172.21.15.22:6834","nonce":3787686782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-07-06T01:20:24.257974+0000","modified":"2023-07-06T01:21:37.327615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":22,"addr":"172.21.15.22:6835/612827809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":612827809},{"type":"v1","addr":"172.21.15.22:6835","nonce":612827809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T01:21:42.558 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.557+0000 7f70335ee700 1 -- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f701006ace0 msgr2=0x7f701006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:42.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.557+0000 7f70335ee700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f701006ace0 0x7f701006d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7028009a30 tx=0x7f7028008040 comp rx=0 tx=0).stop 2023-07-06T01:21:42.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.557+0000 7f70335ee700 1 -- 172.21.15.22:0/1246119675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f702c0af2a0 msgr2=0x7f702c15a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:42.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.557+0000 7f70335ee700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f702c0af2a0 0x7f702c15a800 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f702000e9b0 tx=0x7f702000ed70 comp rx=0 tx=0).stop 2023-07-06T01:21:42.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.557+0000 7f70335ee700 1 -- 172.21.15.22:0/1246119675 shutdown_connections 2023-07-06T01:21:42.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.557+0000 7f70335ee700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f701006ace0 0x7f701006d190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.557+0000 7f70335ee700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f702c0cbc10 0x7f702c0b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.559 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.557+0000 7f70335ee700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f702c06a000 0x7f702c153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.557+0000 7f70335ee700 1 --2- 172.21.15.22:0/1246119675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f702c0af2a0 0x7f702c15a800 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.557+0000 7f70335ee700 1 -- 172.21.15.22:0/1246119675 >> 172.21.15.22:0/1246119675 conn(0x7f702c0b1890 msgr2=0x7f702c06e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:42.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.557+0000 7f70335ee700 1 -- 172.21.15.22:0/1246119675 shutdown_connections 2023-07-06T01:21:42.560 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.557+0000 7f70335ee700 1 -- 172.21.15.22:0/1246119675 wait complete. 2023-07-06T01:21:42.560 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2023-07-06T01:21:42.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:21:42.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.694+0000 7fef9840c700 1 Processor -- start 2023-07-06T01:21:42.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.694+0000 7fef9840c700 1 -- start start 2023-07-06T01:21:42.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef9840c700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fef9015c750 0x7fef900cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef9840c700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fef900cdf80 0x7fef9006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef9840c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef900ce400 0x7fef9006b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef9840c700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef900d06f0 con 0x7fef900cdf80 2023-07-06T01:21:42.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef9840c700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef900ceb90 con 0x7fef9015c750 2023-07-06T01:21:42.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef9840c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef900cee70 con 0x7fef900ce400 2023-07-06T01:21:42.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef959a7700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fef900cdf80 0x7fef9006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef959a7700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fef900cdf80 0x7fef9006b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47996/0 (socket says 172.21.15.22:47996) 2023-07-06T01:21:42.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef959a7700 1 -- 172.21.15.22:0/88304067 learned_addr learned my addr 172.21.15.22:0/88304067 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:42.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef969a9700 1 --2- 172.21.15.22:0/88304067 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef900ce400 0x7fef9006b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef961a8700 1 --2- 172.21.15.22:0/88304067 >> v2:172.21.15.103:3300/0 conn(0x7fef9015c750 0x7fef900cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef959a7700 1 -- 172.21.15.22:0/88304067 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef900ce400 msgr2=0x7fef9006b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:42.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef959a7700 1 --2- 172.21.15.22:0/88304067 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef900ce400 0x7fef9006b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef959a7700 1 -- 172.21.15.22:0/88304067 >> v2:172.21.15.103:3300/0 conn(0x7fef9015c750 msgr2=0x7fef900cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:42.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef959a7700 1 --2- 172.21.15.22:0/88304067 >> v2:172.21.15.103:3300/0 conn(0x7fef9015c750 0x7fef900cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef959a7700 1 -- 172.21.15.22:0/88304067 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef900cf150 con 0x7fef900cdf80 2023-07-06T01:21:42.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef961a8700 1 --2- 172.21.15.22:0/88304067 >> v2:172.21.15.103:3300/0 conn(0x7fef9015c750 0x7fef900cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:42.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef969a9700 1 --2- 172.21.15.22:0/88304067 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef900ce400 0x7fef9006b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:42.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.695+0000 7fef959a7700 1 --2- 172.21.15.22:0/88304067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef900cdf80 0x7fef9006b110 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fef90070f00 tx=0x7fef8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4636edb06aca26 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:42.700 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.696+0000 7fef951a6700 1 -- 172.21.15.22:0/88304067 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00b870 con 0x7fef900cdf80 2023-07-06T01:21:42.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.696+0000 7fef951a6700 1 -- 172.21.15.22:0/88304067 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c00b9d0 con 0x7fef900cdf80 2023-07-06T01:21:42.701 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.696+0000 7fef951a6700 1 -- 172.21.15.22:0/88304067 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00bd60 con 0x7fef900cdf80 2023-07-06T01:21:42.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.696+0000 7fef9840c700 1 -- 172.21.15.22:0/88304067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef900cdf80 msgr2=0x7fef9006b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:42.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.696+0000 7fef9840c700 1 --2- 172.21.15.22:0/88304067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef900cdf80 0x7fef9006b110 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fef90070f00 tx=0x7fef8c009bd0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.696+0000 7fef9840c700 1 -- 172.21.15.22:0/88304067 shutdown_connections 2023-07-06T01:21:42.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.696+0000 7fef9840c700 1 --2- 172.21.15.22:0/88304067 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef900ce400 0x7fef9006b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.696+0000 7fef9840c700 1 --2- 172.21.15.22:0/88304067 >> v2:172.21.15.103:3300/0 conn(0x7fef9015c750 0x7fef900cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.696+0000 7fef9840c700 1 --2- 172.21.15.22:0/88304067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef900cdf80 0x7fef9006b110 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.696+0000 7fef9840c700 1 -- 172.21.15.22:0/88304067 >> 172.21.15.22:0/88304067 conn(0x7fef900999f0 msgr2=0x7fef90099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:42.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.696+0000 7fef9840c700 1 -- 172.21.15.22:0/88304067 shutdown_connections 2023-07-06T01:21:42.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.696+0000 7fef9840c700 1 -- 172.21.15.22:0/88304067 wait complete. 2023-07-06T01:21:42.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.697+0000 7fef9840c700 1 Processor -- start 2023-07-06T01:21:42.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.698+0000 7fef9840c700 1 -- start start 2023-07-06T01:21:42.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.698+0000 7fef9840c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef900cdf80 0x7fef901f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.698+0000 7fef9840c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef900ce400 0x7fef90145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.698+0000 7fef9840c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef9015c750 0x7fef90142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.698+0000 7fef9840c700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef90145d70 con 0x7fef900cdf80 2023-07-06T01:21:42.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.698+0000 7fef9840c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef900aff70 con 0x7fef9015c750 2023-07-06T01:21:42.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.698+0000 7fef9840c700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef900b0250 con 0x7fef900ce400 2023-07-06T01:21:42.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.698+0000 7fef961a8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef900cdf80 0x7fef901f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.698+0000 7fef961a8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef900cdf80 0x7fef901f19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48000/0 (socket says 172.21.15.22:48000) 2023-07-06T01:21:42.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.698+0000 7fef961a8700 1 -- 172.21.15.22:0/2520711180 learned_addr learned my addr 172.21.15.22:0/2520711180 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:42.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.699+0000 7fef959a7700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef900ce400 0x7fef90145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.699+0000 7fef969a9700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef9015c750 0x7fef90142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.699+0000 7fef961a8700 1 -- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef900ce400 msgr2=0x7fef90145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:42.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.699+0000 7fef961a8700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef900ce400 0x7fef90145830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.699+0000 7fef961a8700 1 -- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef9015c750 msgr2=0x7fef90142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:42.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.699+0000 7fef961a8700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef9015c750 0x7fef90142dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.699+0000 7fef961a8700 1 -- 172.21.15.22:0/2520711180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c009550 con 0x7fef900cdf80 2023-07-06T01:21:42.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.699+0000 7fef961a8700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef900cdf80 0x7fef901f19c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c7f0 tx=0x7fef8000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:42.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.699+0000 7fef86ffd700 1 -- 172.21.15.22:0/2520711180 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80011800 con 0x7fef900cdf80 2023-07-06T01:21:42.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.699+0000 7fef86ffd700 1 -- 172.21.15.22:0/2520711180 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef80011960 con 0x7fef900cdf80 2023-07-06T01:21:42.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.699+0000 7fef9840c700 1 -- 172.21.15.22:0/2520711180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef900b19a0 con 0x7fef900cdf80 2023-07-06T01:21:42.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.699+0000 7fef86ffd700 1 -- 172.21.15.22:0/2520711180 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80011d10 con 0x7fef900cdf80 2023-07-06T01:21:42.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.699+0000 7fef9840c700 1 -- 172.21.15.22:0/2520711180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef900b1f60 con 0x7fef900cdf80 2023-07-06T01:21:42.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.700+0000 7fef9840c700 1 -- 172.21.15.22:0/2520711180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef900b1800 con 0x7fef900cdf80 2023-07-06T01:21:42.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.700+0000 7fef86ffd700 1 -- 172.21.15.22:0/2520711180 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef80018360 con 0x7fef900cdf80 2023-07-06T01:21:42.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.701+0000 7fef86ffd700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef7406ace0 0x7fef7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:42.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.701+0000 7fef86ffd700 1 -- 172.21.15.22:0/2520711180 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6119+0+0 (secure 0 0 0) 0x7fef80089400 con 0x7fef900cdf80 2023-07-06T01:21:42.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.701+0000 7fef959a7700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef7406ace0 0x7fef7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:42.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.702+0000 7fef959a7700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef7406ace0 0x7fef7406d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009840 tx=0x7fef8c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:42.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.705+0000 7fef86ffd700 1 -- 172.21.15.22:0/2520711180 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fef900b1800 con 0x7fef900cdf80 2023-07-06T01:21:42.948 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.946+0000 7fef9840c700 1 -- 172.21.15.22:0/2520711180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef900b1800 con 0x7fef900cdf80 2023-07-06T01:21:42.949 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.947+0000 7fef86ffd700 1 -- 172.21.15.22:0/2520711180 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4672 (secure 0 0 0) 0x7fef900b1800 con 0x7fef900cdf80 2023-07-06T01:21:42.949 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:21:42.949 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/219695916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":219695916},{"type":"v1","addr":"172.21.15.103:6836","nonce":219695916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3102932229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":3102932229},{"type":"v1","addr":"172.21.15.103:6837","nonce":3102932229}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3787686782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3787686782},{"type":"v1","addr":"172.21.15.22:6834","nonce":3787686782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-07-06T01:20:24.257974+0000","modified":"2023-07-06T01:21:37.327615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":22,"addr":"172.21.15.22:6835/612827809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":612827809},{"type":"v1","addr":"172.21.15.22:6835","nonce":612827809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T01:21:42.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.949+0000 7fef9840c700 1 -- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef7406ace0 msgr2=0x7fef7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:42.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.949+0000 7fef9840c700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef7406ace0 0x7fef7406d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009840 tx=0x7fef8c008030 comp rx=0 tx=0).stop 2023-07-06T01:21:42.951 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.949+0000 7fef9840c700 1 -- 172.21.15.22:0/2520711180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef900cdf80 msgr2=0x7fef901f19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:42.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.949+0000 7fef9840c700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef900cdf80 0x7fef901f19c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c7f0 tx=0x7fef8000cbb0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.949+0000 7fef9840c700 1 -- 172.21.15.22:0/2520711180 shutdown_connections 2023-07-06T01:21:42.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.949+0000 7fef9840c700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef7406ace0 0x7fef7406d190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.949+0000 7fef9840c700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef9015c750 0x7fef90142dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.949+0000 7fef9840c700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fef900cdf80 0x7fef901f19c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.949+0000 7fef9840c700 1 --2- 172.21.15.22:0/2520711180 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fef900ce400 0x7fef90145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:42.952 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.949+0000 7fef9840c700 1 -- 172.21.15.22:0/2520711180 >> 172.21.15.22:0/2520711180 conn(0x7fef900999f0 msgr2=0x7fef90158020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:42.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.949+0000 7fef9840c700 1 -- 172.21.15.22:0/2520711180 shutdown_connections 2023-07-06T01:21:42.953 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:42.949+0000 7fef9840c700 1 -- 172.21.15.22:0/2520711180 wait complete. 2023-07-06T01:21:42.953 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 24 2023-07-06T01:21:42.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:21:43.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.086+0000 7f7fa58bc700 1 Processor -- start 2023-07-06T01:21:43.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.086+0000 7f7fa58bc700 1 -- start start 2023-07-06T01:21:43.089 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.086+0000 7f7fa58bc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7fa015a6c0 0x7f7fa015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.086+0000 7f7fa58bc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7fa0071980 0x7f7fa006a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.086+0000 7f7fa58bc700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa015f0b0 con 0x7f7fa006a540 2023-07-06T01:21:43.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.086+0000 7f7fa58bc700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa006f260 con 0x7f7fa0071980 2023-07-06T01:21:43.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.086+0000 7f7fa58bc700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa006a980 con 0x7f7fa015a6c0 2023-07-06T01:21:43.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.087+0000 7f7f9e7fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7fa0071980 0x7f7fa006a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.087+0000 7f7f9e7fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7fa0071980 0x7f7fa006a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48010/0 (socket says 172.21.15.22:48010) 2023-07-06T01:21:43.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.087+0000 7f7f9e7fc700 1 -- 172.21.15.22:0/2215131183 learned_addr learned my addr 172.21.15.22:0/2215131183 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:43.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.087+0000 7f7f9effd700 1 --2- 172.21.15.22:0/2215131183 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7fa015a6c0 0x7f7fa015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.087+0000 7f7f9e7fc700 1 -- 172.21.15.22:0/2215131183 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7fa015a6c0 msgr2=0x7f7fa015eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:43.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.087+0000 7f7f9e7fc700 1 --2- 172.21.15.22:0/2215131183 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7fa015a6c0 0x7f7fa015eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.087+0000 7f7f9e7fc700 1 -- 172.21.15.22:0/2215131183 >> v1:172.21.15.103:6789/0 conn(0x7f7fa006a540 legacy=0x7f7fa00ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:43.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.087+0000 7f7f9e7fc700 1 -- 172.21.15.22:0/2215131183 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa015aa90 con 0x7f7fa0071980 2023-07-06T01:21:43.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.087+0000 7f7f9e7fc700 1 --2- 172.21.15.22:0/2215131183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa0071980 0x7f7fa006a000 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7f900098f0 tx=0x7f7f90009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfc990e3959c9c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:43.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.087+0000 7f7f9dffb700 1 -- 172.21.15.22:0/2215131183 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90004230 con 0x7f7fa0071980 2023-07-06T01:21:43.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.087+0000 7f7f9dffb700 1 -- 172.21.15.22:0/2215131183 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f90004390 con 0x7f7fa0071980 2023-07-06T01:21:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.087+0000 7f7f9dffb700 1 -- 172.21.15.22:0/2215131183 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f900036a0 con 0x7f7fa0071980 2023-07-06T01:21:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.088+0000 7f7fa58bc700 1 -- 172.21.15.22:0/2215131183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa0071980 msgr2=0x7f7fa006a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.088+0000 7f7fa58bc700 1 --2- 172.21.15.22:0/2215131183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa0071980 0x7f7fa006a000 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7f900098f0 tx=0x7f7f90009c00 comp rx=0 tx=0).stop 2023-07-06T01:21:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.088+0000 7f7fa58bc700 1 -- 172.21.15.22:0/2215131183 shutdown_connections 2023-07-06T01:21:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.088+0000 7f7fa58bc700 1 --2- 172.21.15.22:0/2215131183 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7fa015a6c0 0x7f7fa015eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.088+0000 7f7fa58bc700 1 --2- 172.21.15.22:0/2215131183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa0071980 0x7f7fa006a000 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.088+0000 7f7fa58bc700 1 -- 172.21.15.22:0/2215131183 >> 172.21.15.22:0/2215131183 conn(0x7f7fa00b1890 msgr2=0x7f7fa00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:43.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.088+0000 7f7fa58bc700 1 -- 172.21.15.22:0/2215131183 shutdown_connections 2023-07-06T01:21:43.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.088+0000 7f7fa58bc700 1 -- 172.21.15.22:0/2215131183 wait complete. 2023-07-06T01:21:43.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.089+0000 7f7fa58bc700 1 Processor -- start 2023-07-06T01:21:43.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.089+0000 7f7fa58bc700 1 -- start start 2023-07-06T01:21:43.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7fa58bc700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7fa006a540 0x7f7fa0153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7fa58bc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa015a6c0 0x7f7fa0154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7fa58bc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa0154f50 0x7f7fa01e40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7fa58bc700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01f3760 con 0x7f7fa0154f50 2023-07-06T01:21:43.095 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7fa58bc700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01e5880 con 0x7f7fa015a6c0 2023-07-06T01:21:43.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7fa58bc700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01553d0 con 0x7f7fa006a540 2023-07-06T01:21:43.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7f9f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa0154f50 0x7f7fa01e40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7f9f7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa0154f50 0x7f7fa01e40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48018/0 (socket says 172.21.15.22:48018) 2023-07-06T01:21:43.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7f9f7fe700 1 -- 172.21.15.22:0/1934668613 learned_addr learned my addr 172.21.15.22:0/1934668613 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:43.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7f9effd700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7fa006a540 0x7f7fa0153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7f9e7fc700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa015a6c0 0x7f7fa0154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.096 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7f9f7fe700 1 -- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7fa006a540 msgr2=0x7f7fa0153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7f9f7fe700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7fa006a540 0x7f7fa0153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7f9f7fe700 1 -- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa015a6c0 msgr2=0x7f7fa0154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7f9f7fe700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa015a6c0 0x7f7fa0154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7f9f7fe700 1 -- 172.21.15.22:0/1934668613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f90009550 con 0x7f7fa0154f50 2023-07-06T01:21:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7f9e7fc700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa015a6c0 0x7f7fa0154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.090+0000 7f7f9f7fe700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa0154f50 0x7f7fa01e40f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800cc10 tx=0x7f7f9800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.091+0000 7f7f87fff700 1 -- 172.21.15.22:0/1934668613 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800ab10 con 0x7f7fa0154f50 2023-07-06T01:21:43.097 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.091+0000 7f7f87fff700 1 -- 172.21.15.22:0/1934668613 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9800ac70 con 0x7f7fa0154f50 2023-07-06T01:21:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.091+0000 7f7fa58bc700 1 -- 172.21.15.22:0/1934668613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa00c8450 con 0x7f7fa0154f50 2023-07-06T01:21:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.091+0000 7f7f87fff700 1 -- 172.21.15.22:0/1934668613 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98012420 con 0x7f7fa0154f50 2023-07-06T01:21:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.091+0000 7f7fa58bc700 1 -- 172.21.15.22:0/1934668613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa01e4db0 con 0x7f7fa0154f50 2023-07-06T01:21:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.092+0000 7f7fa58bc700 1 -- 172.21.15.22:0/1934668613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa00744b0 con 0x7f7fa0154f50 2023-07-06T01:21:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.092+0000 7f7f87fff700 1 -- 172.21.15.22:0/1934668613 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f98012580 con 0x7f7fa0154f50 2023-07-06T01:21:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.093+0000 7f7f87fff700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f8c06ace0 0x7f7f8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.098 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.093+0000 7f7f87fff700 1 -- 172.21.15.22:0/1934668613 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6119+0+0 (secure 0 0 0) 0x7f7f98089830 con 0x7f7fa0154f50 2023-07-06T01:21:43.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.093+0000 7f7f9effd700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f8c06ace0 0x7f7f8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.093+0000 7f7f9effd700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f8c06ace0 0x7f7f8c06d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7f94006fd0 tx=0x7f7f94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:43.099 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.097+0000 7f7f87fff700 1 -- 172.21.15.22:0/1934668613 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7f980548a0 con 0x7f7fa0154f50 2023-07-06T01:21:43.302 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.301+0000 7f7fa58bc700 1 -- 172.21.15.22:0/1934668613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fa00c85e0 con 0x7f7fa0154f50 2023-07-06T01:21:43.304 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.302+0000 7f7f87fff700 1 -- 172.21.15.22:0/1934668613 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18339 (secure 0 0 0) 0x7f7f98057ec0 con 0x7f7fa0154f50 2023-07-06T01:21:43.304 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:21:43.304 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":38,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:21:41.176818+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T01:20:22.043370+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T01:20:23.225298+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:21:43.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.304+0000 7f7fa58bc700 1 -- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f8c06ace0 msgr2=0x7f7f8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.304+0000 7f7fa58bc700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f8c06ace0 0x7f7f8c06d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7f94006fd0 tx=0x7f7f94008040 comp rx=0 tx=0).stop 2023-07-06T01:21:43.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.304+0000 7f7fa58bc700 1 -- 172.21.15.22:0/1934668613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa0154f50 msgr2=0x7f7fa01e40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.306 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.304+0000 7f7fa58bc700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa0154f50 0x7f7fa01e40f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800cc10 tx=0x7f7f9800cf20 comp rx=0 tx=0).stop 2023-07-06T01:21:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.304+0000 7f7fa58bc700 1 -- 172.21.15.22:0/1934668613 shutdown_connections 2023-07-06T01:21:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.304+0000 7f7fa58bc700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7fa0154f50 0x7f7fa01e40f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.304+0000 7f7fa58bc700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7fa006a540 0x7f7fa0153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.304+0000 7f7fa58bc700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7f8c06ace0 0x7f7f8c06d190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.304+0000 7f7fa58bc700 1 --2- 172.21.15.22:0/1934668613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7fa015a6c0 0x7f7fa0154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.304+0000 7f7fa58bc700 1 -- 172.21.15.22:0/1934668613 >> 172.21.15.22:0/1934668613 conn(0x7f7fa00b1890 msgr2=0x7f7fa00cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.305+0000 7f7fa58bc700 1 -- 172.21.15.22:0/1934668613 shutdown_connections 2023-07-06T01:21:43.308 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.305+0000 7f7fa58bc700 1 -- 172.21.15.22:0/1934668613 wait complete. 2023-07-06T01:21:43.318 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T01:21:43.444 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.442+0000 7ff6ff7ab700 1 Processor -- start 2023-07-06T01:21:43.445 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.442+0000 7ff6ff7ab700 1 -- start start 2023-07-06T01:21:43.449 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6ff7ab700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6f815a6c0 0x7ff6f815eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6ff7ab700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff6f8071980 0x7ff6f806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6ff7ab700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff6f806a540 0x7ff6f80ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6ff7ab700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f806f260 con 0x7ff6f806a540 2023-07-06T01:21:43.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6ff7ab700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f815f0b0 con 0x7ff6f8071980 2023-07-06T01:21:43.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6ff7ab700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f806a980 con 0x7ff6f815a6c0 2023-07-06T01:21:43.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6fdd48700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff6f806a540 0x7ff6f80ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.450 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6fdd48700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff6f806a540 0x7ff6f80ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48026/0 (socket says 172.21.15.22:48026) 2023-07-06T01:21:43.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6fdd48700 1 -- 172.21.15.22:0/1917407079 learned_addr learned my addr 172.21.15.22:0/1917407079 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:43.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6fd547700 1 --2- 172.21.15.22:0/1917407079 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6f815a6c0 0x7ff6f815eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6fdd48700 1 -- 172.21.15.22:0/1917407079 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6f815a6c0 msgr2=0x7ff6f815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:43.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6fcd46700 1 --2- 172.21.15.22:0/1917407079 >> v2:172.21.15.103:3300/0 conn(0x7ff6f8071980 0x7ff6f806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6fdd48700 1 --2- 172.21.15.22:0/1917407079 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6f815a6c0 0x7ff6f815eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6fdd48700 1 -- 172.21.15.22:0/1917407079 >> v2:172.21.15.103:3300/0 conn(0x7ff6f8071980 msgr2=0x7ff6f806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:43.451 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6fdd48700 1 --2- 172.21.15.22:0/1917407079 >> v2:172.21.15.103:3300/0 conn(0x7ff6f8071980 0x7ff6f806a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.443+0000 7ff6fdd48700 1 -- 172.21.15.22:0/1917407079 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f815aa90 con 0x7ff6f806a540 2023-07-06T01:21:43.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.444+0000 7ff6fdd48700 1 --2- 172.21.15.22:0/1917407079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f806a540 0x7ff6f80ca2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80cd760 tx=0x7ff6f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1250817a81514c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:43.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.444+0000 7ff6effff700 1 -- 172.21.15.22:0/1917407079 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f000e070 con 0x7ff6f806a540 2023-07-06T01:21:43.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.444+0000 7ff6effff700 1 -- 172.21.15.22:0/1917407079 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f000b9c0 con 0x7ff6f806a540 2023-07-06T01:21:43.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.444+0000 7ff6effff700 1 -- 172.21.15.22:0/1917407079 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f000bd70 con 0x7ff6f806a540 2023-07-06T01:21:43.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.444+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/1917407079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f806a540 msgr2=0x7ff6f80ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.452 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.444+0000 7ff6ff7ab700 1 --2- 172.21.15.22:0/1917407079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f806a540 0x7ff6f80ca2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80cd760 tx=0x7ff6f000b330 comp rx=0 tx=0).stop 2023-07-06T01:21:43.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.444+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/1917407079 shutdown_connections 2023-07-06T01:21:43.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.444+0000 7ff6ff7ab700 1 --2- 172.21.15.22:0/1917407079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f806a540 0x7ff6f80ca2f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.444+0000 7ff6ff7ab700 1 --2- 172.21.15.22:0/1917407079 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6f815a6c0 0x7ff6f815eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.444+0000 7ff6ff7ab700 1 --2- 172.21.15.22:0/1917407079 >> v2:172.21.15.103:3300/0 conn(0x7ff6f8071980 0x7ff6f806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.444+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/1917407079 >> 172.21.15.22:0/1917407079 conn(0x7ff6f80b1890 msgr2=0x7ff6f80b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:43.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.444+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/1917407079 shutdown_connections 2023-07-06T01:21:43.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.444+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/1917407079 wait complete. 2023-07-06T01:21:43.453 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.446+0000 7ff6ff7ab700 1 Processor -- start 2023-07-06T01:21:43.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.446+0000 7ff6ff7ab700 1 -- start start 2023-07-06T01:21:43.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.446+0000 7ff6ff7ab700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6f806a540 0x7ff6f80c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.446+0000 7ff6ff7ab700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f815a6c0 0x7ff6f80c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.454 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.446+0000 7ff6ff7ab700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6f81f2fd0 0x7ff6f80c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.446+0000 7ff6ff7ab700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f80c6460 con 0x7ff6f815a6c0 2023-07-06T01:21:43.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.446+0000 7ff6ff7ab700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f80c35b0 con 0x7ff6f806a540 2023-07-06T01:21:43.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.446+0000 7ff6ff7ab700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f80c3890 con 0x7ff6f81f2fd0 2023-07-06T01:21:43.455 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.446+0000 7ff6fcd46700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f815a6c0 0x7ff6f80c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6fcd46700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f815a6c0 0x7ff6f80c5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48030/0 (socket says 172.21.15.22:48030) 2023-07-06T01:21:43.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6fcd46700 1 -- 172.21.15.22:0/3182266573 learned_addr learned my addr 172.21.15.22:0/3182266573 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:43.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6fd547700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6f806a540 0x7ff6f80c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.456 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6fdd48700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6f81f2fd0 0x7ff6f80c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6fcd46700 1 -- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6f81f2fd0 msgr2=0x7ff6f80c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6fcd46700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6f81f2fd0 0x7ff6f80c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6fcd46700 1 -- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6f806a540 msgr2=0x7ff6f80c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.457 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6fcd46700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6f806a540 0x7ff6f80c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6fcd46700 1 -- 172.21.15.22:0/3182266573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f0009b30 con 0x7ff6f815a6c0 2023-07-06T01:21:43.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6fdd48700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6f81f2fd0 0x7ff6f80c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:43.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6fd547700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6f806a540 0x7ff6f80c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:43.458 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6fcd46700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f815a6c0 0x7ff6f80c5ac0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800c7f0 tx=0x7ff6e800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:43.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6edffb700 1 -- 172.21.15.22:0/3182266573 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8011800 con 0x7ff6f815a6c0 2023-07-06T01:21:43.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6edffb700 1 -- 172.21.15.22:0/3182266573 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e8011960 con 0x7ff6f815a6c0 2023-07-06T01:21:43.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/3182266573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f80c3b70 con 0x7ff6f815a6c0 2023-07-06T01:21:43.459 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.447+0000 7ff6edffb700 1 -- 172.21.15.22:0/3182266573 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8011d10 con 0x7ff6f815a6c0 2023-07-06T01:21:43.460 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.448+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/3182266573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f80c5660 con 0x7ff6f815a6c0 2023-07-06T01:21:43.461 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.448+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/3182266573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f80c5850 con 0x7ff6f815a6c0 2023-07-06T01:21:43.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.449+0000 7ff6edffb700 1 -- 172.21.15.22:0/3182266573 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff6e8018360 con 0x7ff6f815a6c0 2023-07-06T01:21:43.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.449+0000 7ff6edffb700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff6dc06ace0 0x7ff6dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.462 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.449+0000 7ff6edffb700 1 -- 172.21.15.22:0/3182266573 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6119+0+0 (secure 0 0 0) 0x7ff6e8089430 con 0x7ff6f815a6c0 2023-07-06T01:21:43.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.449+0000 7ff6fd547700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff6dc06ace0 0x7ff6dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.450+0000 7ff6fd547700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff6dc06ace0 0x7ff6dc06d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80b5d90 tx=0x7ff6f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:43.463 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.453+0000 7ff6edffb700 1 -- 172.21.15.22:0/3182266573 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff6e80544a0 con 0x7ff6f815a6c0 2023-07-06T01:21:43.699 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.698+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/3182266573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff6f80c4090 con 0x7ff6f815a6c0 2023-07-06T01:21:43.700 INFO:tasks.ceph.mon.a.smithi022.stderr:2023-07-06T01:21:43.698+0000 7ff5d4bc7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T01:21:43.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.704+0000 7ff6edffb700 1 -- 172.21.15.22:0/3182266573 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff6f80c4090 con 0x7ff6f815a6c0 2023-07-06T01:21:43.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.706+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff6dc06ace0 msgr2=0x7ff6dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.706+0000 7ff6ff7ab700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff6dc06ace0 0x7ff6dc06d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80b5d90 tx=0x7ff6f4008040 comp rx=0 tx=0).stop 2023-07-06T01:21:43.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.706+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/3182266573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f815a6c0 msgr2=0x7ff6f80c5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.706+0000 7ff6ff7ab700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f815a6c0 0x7ff6f80c5ac0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800c7f0 tx=0x7ff6e800cbb0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.707+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/3182266573 shutdown_connections 2023-07-06T01:21:43.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.707+0000 7ff6ff7ab700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6f81f2fd0 0x7ff6f80c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.707+0000 7ff6ff7ab700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff6dc06ace0 0x7ff6dc06d190 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.707+0000 7ff6ff7ab700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6f806a540 0x7ff6f80c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.707+0000 7ff6ff7ab700 1 --2- 172.21.15.22:0/3182266573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6f815a6c0 0x7ff6f80c5ac0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.707+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/3182266573 >> 172.21.15.22:0/3182266573 conn(0x7ff6f80b1890 msgr2=0x7ff6f80afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:43.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.707+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/3182266573 shutdown_connections 2023-07-06T01:21:43.710 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.707+0000 7ff6ff7ab700 1 -- 172.21.15.22:0/3182266573 wait complete. 2023-07-06T01:21:43.711 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T01:21:43.721 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T01:21:43.747 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2023-07-06T01:21:43.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.848+0000 7f614c09a700 1 Processor -- start 2023-07-06T01:21:43.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.848+0000 7f614c09a700 1 -- start start 2023-07-06T01:21:43.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.849+0000 7f614c09a700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f614406a710 0x7f61440ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.849+0000 7f614c09a700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440ca2b0 con 0x7f614415cb50 2023-07-06T01:21:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.849+0000 7f614c09a700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440cfc90 con 0x7f61440cf090 2023-07-06T01:21:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.849+0000 7f614c09a700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61440af440 con 0x7f614406a710 2023-07-06T01:21:43.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.849+0000 7f6149e36700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f614415cb50 0x7f614406a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38792/0 (socket says 172.21.15.22:38792) 2023-07-06T01:21:43.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.849+0000 7f6149e36700 1 -- 172.21.15.22:0/2831692998 learned_addr learned my addr 172.21.15.22:0/2831692998 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:43.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.849+0000 7f6149635700 1 --2- 172.21.15.22:0/2831692998 >> v2:172.21.15.103:3300/0 conn(0x7f614406a710 0x7f61440ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.849+0000 7f6148e34700 1 -- 172.21.15.22:0/2831692998 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452294198 0 0) 0x7f61440ca2b0 con 0x7f614415cb50 2023-07-06T01:21:43.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.849+0000 7f6148e34700 1 -- 172.21.15.22:0/2831692998 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61300035b0 con 0x7f614415cb50 2023-07-06T01:21:43.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.849+0000 7f6148e34700 1 -- 172.21.15.22:0/2831692998 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908869630 0 0) 0x7f61440cfc90 con 0x7f61440cf090 2023-07-06T01:21:43.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.849+0000 7f6148e34700 1 -- 172.21.15.22:0/2831692998 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61440ca2b0 con 0x7f61440cf090 2023-07-06T01:21:43.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.850+0000 7f6149635700 1 -- 172.21.15.22:0/2831692998 >> v1:172.21.15.103:6789/0 conn(0x7f61440cf090 legacy=0x7f6144160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.850+0000 7f6149635700 1 -- 172.21.15.22:0/2831692998 >> v1:172.21.15.22:6789/0 conn(0x7f614415cb50 legacy=0x7f614406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.850+0000 7f6149635700 1 -- 172.21.15.22:0/2831692998 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61440cff30 con 0x7f614406a710 2023-07-06T01:21:43.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.850+0000 7f6149635700 1 --2- 172.21.15.22:0/2831692998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614406a710 0x7f61440ceb50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6134009840 tx=0x7f6134009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bae87b7afe775c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.850+0000 7f6148e34700 1 -- 172.21.15.22:0/2831692998 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134011830 con 0x7f614406a710 2023-07-06T01:21:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.850+0000 7f6148e34700 1 -- 172.21.15.22:0/2831692998 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6134011990 con 0x7f614406a710 2023-07-06T01:21:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.850+0000 7f6148e34700 1 -- 172.21.15.22:0/2831692998 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134011d00 con 0x7f614406a710 2023-07-06T01:21:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.851+0000 7f614c09a700 1 -- 172.21.15.22:0/2831692998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614406a710 msgr2=0x7f61440ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.851+0000 7f614c09a700 1 --2- 172.21.15.22:0/2831692998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614406a710 0x7f61440ceb50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6134009840 tx=0x7f6134009b50 comp rx=0 tx=0).stop 2023-07-06T01:21:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.851+0000 7f614c09a700 1 -- 172.21.15.22:0/2831692998 shutdown_connections 2023-07-06T01:21:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.851+0000 7f614c09a700 1 --2- 172.21.15.22:0/2831692998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614406a710 0x7f61440ceb50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.851+0000 7f614c09a700 1 -- 172.21.15.22:0/2831692998 >> 172.21.15.22:0/2831692998 conn(0x7f614406b110 msgr2=0x7f614406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.851+0000 7f614c09a700 1 -- 172.21.15.22:0/2831692998 shutdown_connections 2023-07-06T01:21:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.851+0000 7f614c09a700 1 -- 172.21.15.22:0/2831692998 wait complete. 2023-07-06T01:21:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.852+0000 7f614c09a700 1 Processor -- start 2023-07-06T01:21:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.852+0000 7f614c09a700 1 -- start start 2023-07-06T01:21:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f614c09a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614406a710 0x7f614415e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f614c09a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61440cf090 0x7f61440b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f614c09a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614415cb50 0x7f61440c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f614c09a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61440c3dc0 con 0x7f614406a710 2023-07-06T01:21:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f614c09a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614415ecd0 con 0x7f614415cb50 2023-07-06T01:21:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f614c09a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61440b54b0 con 0x7f61440cf090 2023-07-06T01:21:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f6149e36700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614406a710 0x7f614415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f6149e36700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614406a710 0x7f614415e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48054/0 (socket says 172.21.15.22:48054) 2023-07-06T01:21:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f6149e36700 1 -- 172.21.15.22:0/3864506424 learned_addr learned my addr 172.21.15.22:0/3864506424 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f614a637700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614415cb50 0x7f61440c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f6149635700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61440cf090 0x7f61440b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f6149e36700 1 -- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61440cf090 msgr2=0x7f61440b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f6149e36700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61440cf090 0x7f61440b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f6149e36700 1 -- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614415cb50 msgr2=0x7f61440c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f6149e36700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614415cb50 0x7f61440c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f6149e36700 1 -- 172.21.15.22:0/3864506424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61340094a0 con 0x7f614406a710 2023-07-06T01:21:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f6149635700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61440cf090 0x7f61440b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f614a637700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614415cb50 0x7f61440c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f6149e36700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614406a710 0x7f614415e790 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f614000c4d0 tx=0x7f614000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f613a7fc700 1 -- 172.21.15.22:0/3864506424 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6140005d70 con 0x7f614406a710 2023-07-06T01:21:43.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.853+0000 7f613a7fc700 1 -- 172.21.15.22:0/3864506424 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61400070d0 con 0x7f614406a710 2023-07-06T01:21:43.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.854+0000 7f614c09a700 1 -- 172.21.15.22:0/3864506424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61440c3790 con 0x7f614406a710 2023-07-06T01:21:43.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.854+0000 7f613a7fc700 1 -- 172.21.15.22:0/3864506424 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6140007460 con 0x7f614406a710 2023-07-06T01:21:43.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.854+0000 7f614c09a700 1 -- 172.21.15.22:0/3864506424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61440c39e0 con 0x7f614406a710 2023-07-06T01:21:43.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.855+0000 7f614c09a700 1 -- 172.21.15.22:0/3864506424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61440c3f00 con 0x7f614406a710 2023-07-06T01:21:43.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.855+0000 7f613a7fc700 1 -- 172.21.15.22:0/3864506424 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f61400075c0 con 0x7f614406a710 2023-07-06T01:21:43.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.855+0000 7f613a7fc700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f612806ace0 0x7f612806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:43.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.855+0000 7f613a7fc700 1 -- 172.21.15.22:0/3864506424 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f6140088a30 con 0x7f614406a710 2023-07-06T01:21:43.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.856+0000 7f6149635700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f612806ace0 0x7f612806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:43.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.856+0000 7f6149635700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f612806ace0 0x7f612806d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f614406ab00 tx=0x7f613401c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:43.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:43.860+0000 7f613a7fc700 1 -- 172.21.15.22:0/3864506424 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f61440c3f00 con 0x7f614406a710 2023-07-06T01:21:44.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.104+0000 7f614c09a700 1 -- 172.21.15.22:0/3864506424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f61440c3f00 con 0x7f614406a710 2023-07-06T01:21:44.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.706+0000 7f613a7fc700 1 -- 172.21.15.22:0/3864506424 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6140053a70 con 0x7f614406a710 2023-07-06T01:21:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.709+0000 7f614c09a700 1 -- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f612806ace0 msgr2=0x7f612806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.709+0000 7f614c09a700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f612806ace0 0x7f612806d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f614406ab00 tx=0x7f613401c040 comp rx=0 tx=0).stop 2023-07-06T01:21:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.709+0000 7f614c09a700 1 -- 172.21.15.22:0/3864506424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614406a710 msgr2=0x7f614415e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.709+0000 7f614c09a700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614406a710 0x7f614415e790 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f614000c4d0 tx=0x7f614000c890 comp rx=0 tx=0).stop 2023-07-06T01:21:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.709+0000 7f614c09a700 1 -- 172.21.15.22:0/3864506424 shutdown_connections 2023-07-06T01:21:44.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.709+0000 7f614c09a700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f612806ace0 0x7f612806d190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.709+0000 7f614c09a700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f614415cb50 0x7f61440c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.709+0000 7f614c09a700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f614406a710 0x7f614415e790 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.709+0000 7f614c09a700 1 --2- 172.21.15.22:0/3864506424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61440cf090 0x7f61440b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.709+0000 7f614c09a700 1 -- 172.21.15.22:0/3864506424 >> 172.21.15.22:0/3864506424 conn(0x7f614406b110 msgr2=0x7f61440cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.709+0000 7f614c09a700 1 -- 172.21.15.22:0/3864506424 shutdown_connections 2023-07-06T01:21:44.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.709+0000 7f614c09a700 1 -- 172.21.15.22:0/3864506424 wait complete. 2023-07-06T01:21:44.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T01:21:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.903+0000 7f6a005df700 1 Processor -- start 2023-07-06T01:21:44.905 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.903+0000 7f6a005df700 1 -- start start 2023-07-06T01:21:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.904+0000 7f6a005df700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69f806a710 0x7f69f80cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.904+0000 7f6a005df700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f80cc4b0 con 0x7f69f815f100 2023-07-06T01:21:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.904+0000 7f6a005df700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f815e3d0 con 0x7f69f815a780 2023-07-06T01:21:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.904+0000 7f6a005df700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f815ec70 con 0x7f69f806a710 2023-07-06T01:21:44.906 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.904+0000 7f69feb7c700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f69f815f100 0x7f69f80d0130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38812/0 (socket says 172.21.15.22:38812) 2023-07-06T01:21:44.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.904+0000 7f69feb7c700 1 -- 172.21.15.22:0/641095569 learned_addr learned my addr 172.21.15.22:0/641095569 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:44.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.904+0000 7f69fdb7a700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69f806a710 0x7f69f80cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:44.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.904+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5897961 0 0) 0x7f69f0001490 con 0x7f69f815f100 2023-07-06T01:21:44.907 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.904+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e40035b0 con 0x7f69f815f100 2023-07-06T01:21:44.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 198968508 0 0) 0x7f69e40035b0 con 0x7f69f815f100 2023-07-06T01:21:44.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69f806a710 msgr2=0x7f69f80cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:44.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69f806a710 0x7f69f80cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:44.909 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 >> v1:172.21.15.103:6789/0 conn(0x7f69f815a780 legacy=0x7f69f806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:44.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f80d0920 con 0x7f69f815f100 2023-07-06T01:21:44.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f69f0001b80 con 0x7f69f815f100 2023-07-06T01:21:44.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 >> v1:172.21.15.22:6789/0 conn(0x7f69f815f100 legacy=0x7f69f80d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:44.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e4003a30 0x7f69e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:44.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69f806a710 0x7f69e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:44.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e4004d90 0x7f69e4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:44.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f815ec70 con 0x7f69e4003a30 2023-07-06T01:21:44.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e400ba10 con 0x7f69e4004d90 2023-07-06T01:21:44.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e400bba0 con 0x7f69f806a710 2023-07-06T01:21:44.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fe37b700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e4003a30 0x7f69e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:44.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69feb7c700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e4004d90 0x7f69e4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:44.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fdb7a700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69f806a710 0x7f69e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:44.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fe37b700 1 -- 172.21.15.22:0/641095569 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69f806a710 msgr2=0x7f69e4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:44.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fe37b700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69f806a710 0x7f69e4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:44.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fe37b700 1 -- 172.21.15.22:0/641095569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e4004d90 msgr2=0x7f69e4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:44.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fe37b700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e4004d90 0x7f69e4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:44.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fe37b700 1 -- 172.21.15.22:0/641095569 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e40035b0 con 0x7f69e4003a30 2023-07-06T01:21:44.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69feb7c700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69e4004d90 0x7f69e4005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:44.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fdb7a700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69f806a710 0x7f69e4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:44.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fe37b700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e4003a30 0x7f69e4003e00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f69e80018d0 tx=0x7f69e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180d2f3832f9ea28 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e800c690 con 0x7f69e4003a30 2023-07-06T01:21:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e800c810 con 0x7f69e4003a30 2023-07-06T01:21:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.905+0000 7f69fd379700 1 -- 172.21.15.22:0/641095569 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e800cbc0 con 0x7f69e4003a30 2023-07-06T01:21:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.906+0000 7f6a005df700 1 -- 172.21.15.22:0/641095569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e4003a30 msgr2=0x7f69e4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.906+0000 7f6a005df700 1 --2- 172.21.15.22:0/641095569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69e4003a30 0x7f69e4003e00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f69e80018d0 tx=0x7f69e4005880 comp rx=0 tx=0).stop 2023-07-06T01:21:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.906+0000 7f69feb7c700 1 -- 172.21.15.22:0/641095569 reap_dead start 2023-07-06T01:21:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.906+0000 7f6a005df700 1 -- 172.21.15.22:0/641095569 shutdown_connections 2023-07-06T01:21:44.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.906+0000 7f6a005df700 1 -- 172.21.15.22:0/641095569 >> 172.21.15.22:0/641095569 conn(0x7f69f806b110 msgr2=0x7f69f806b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.906+0000 7f6a005df700 1 -- 172.21.15.22:0/641095569 shutdown_connections 2023-07-06T01:21:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.906+0000 7f6a005df700 1 -- 172.21.15.22:0/641095569 wait complete. 2023-07-06T01:21:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.907+0000 7f6a005df700 1 Processor -- start 2023-07-06T01:21:44.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.907+0000 7f6a005df700 1 -- start start 2023-07-06T01:21:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.908+0000 7f6a005df700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e4003a30 0x7f69f815be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.908+0000 7f6a005df700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f81e4a00 0x7f69f81f0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.908+0000 7f6a005df700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69f81e4e70 0x7f69f81f0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:44.916 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.908+0000 7f6a005df700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f81e4580 con 0x7f69f81e4e70 2023-07-06T01:21:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.908+0000 7f6a005df700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f81f10a0 con 0x7f69f81e4a00 2023-07-06T01:21:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.908+0000 7f6a005df700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f81f1380 con 0x7f69e4003a30 2023-07-06T01:21:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.908+0000 7f69feb7c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69f81e4e70 0x7f69f81f0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.908+0000 7f69feb7c700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69f81e4e70 0x7f69f81f0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48080/0 (socket says 172.21.15.22:48080) 2023-07-06T01:21:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.908+0000 7f69feb7c700 1 -- 172.21.15.22:0/941502420 learned_addr learned my addr 172.21.15.22:0/941502420 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.908+0000 7f69fe37b700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e4003a30 0x7f69f815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:44.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.908+0000 7f69fdb7a700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f81e4a00 0x7f69f81f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.909+0000 7f69feb7c700 1 -- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e4003a30 msgr2=0x7f69f815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.909+0000 7f69feb7c700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e4003a30 0x7f69f815be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.909+0000 7f69feb7c700 1 -- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f81e4a00 msgr2=0x7f69f81f0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.909+0000 7f69feb7c700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f81e4a00 0x7f69f81f0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.909+0000 7f69feb7c700 1 -- 172.21.15.22:0/941502420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f8157950 con 0x7f69f81e4e70 2023-07-06T01:21:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.909+0000 7f69fe37b700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e4003a30 0x7f69f815be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:44.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.909+0000 7f69feb7c700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69f81e4e70 0x7f69f81f0990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f69f80cb290 tx=0x7f69f000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.909+0000 7f69eeffd700 1 -- 172.21.15.22:0/941502420 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f000ebd0 con 0x7f69f81e4e70 2023-07-06T01:21:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.909+0000 7f69eeffd700 1 -- 172.21.15.22:0/941502420 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f000ed30 con 0x7f69f81e4e70 2023-07-06T01:21:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.909+0000 7f6a005df700 1 -- 172.21.15.22:0/941502420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f8157c30 con 0x7f69f81e4e70 2023-07-06T01:21:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.909+0000 7f69eeffd700 1 -- 172.21.15.22:0/941502420 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0013640 con 0x7f69f81e4e70 2023-07-06T01:21:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.909+0000 7f6a005df700 1 -- 172.21.15.22:0/941502420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f81e6510 con 0x7f69f81e4e70 2023-07-06T01:21:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.910+0000 7f6a005df700 1 -- 172.21.15.22:0/941502420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f8158150 con 0x7f69f81e4e70 2023-07-06T01:21:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.910+0000 7f69eeffd700 1 -- 172.21.15.22:0/941502420 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f69f00137a0 con 0x7f69f81e4e70 2023-07-06T01:21:44.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.911+0000 7f69eeffd700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f69dc06ace0 0x7f69dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:44.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.911+0000 7f69eeffd700 1 -- 172.21.15.22:0/941502420 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f69f00898c0 con 0x7f69f81e4e70 2023-07-06T01:21:44.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.911+0000 7f69fe37b700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f69dc06ace0 0x7f69dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:44.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.912+0000 7f69fe37b700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f69dc06ace0 0x7f69dc06d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f69e8014040 tx=0x7f69e8005040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:44.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:44.916+0000 7f69eeffd700 1 -- 172.21.15.22:0/941502420 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f69f8158150 con 0x7f69f81e4e70 2023-07-06T01:21:45.115 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.113+0000 7f6a005df700 1 -- 172.21.15.22:0/941502420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f69f80cc4b0 con 0x7f69f81e4e70 2023-07-06T01:21:45.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.709+0000 7f69eeffd700 1 -- 172.21.15.22:0/941502420 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) v1 ==== 181+0+0 (secure 0 0 0) 0x7f69f80cc4b0 con 0x7f69f81e4e70 2023-07-06T01:21:45.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.714+0000 7f6a005df700 1 -- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f69dc06ace0 msgr2=0x7f69dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.714+0000 7f6a005df700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f69dc06ace0 0x7f69dc06d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f69e8014040 tx=0x7f69e8005040 comp rx=0 tx=0).stop 2023-07-06T01:21:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.714+0000 7f6a005df700 1 -- 172.21.15.22:0/941502420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69f81e4e70 msgr2=0x7f69f81f0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.714+0000 7f6a005df700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69f81e4e70 0x7f69f81f0990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f69f80cb290 tx=0x7f69f000e440 comp rx=0 tx=0).stop 2023-07-06T01:21:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.714+0000 7f6a005df700 1 -- 172.21.15.22:0/941502420 shutdown_connections 2023-07-06T01:21:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.714+0000 7f6a005df700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69f81e4e70 0x7f69f81f0990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.714+0000 7f6a005df700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e4003a30 0x7f69f815be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.714+0000 7f6a005df700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f69dc06ace0 0x7f69dc06d190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.714+0000 7f6a005df700 1 --2- 172.21.15.22:0/941502420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69f81e4a00 0x7f69f81f0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.714+0000 7f6a005df700 1 -- 172.21.15.22:0/941502420 >> 172.21.15.22:0/941502420 conn(0x7f69f806b110 msgr2=0x7f69f80b4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.715+0000 7f6a005df700 1 -- 172.21.15.22:0/941502420 shutdown_connections 2023-07-06T01:21:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.715+0000 7f6a005df700 1 -- 172.21.15.22:0/941502420 wait complete. 2023-07-06T01:21:45.718 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2023-07-06T01:21:45.734 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T01:21:45.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.868+0000 7ff61284e700 1 Processor -- start 2023-07-06T01:21:45.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.868+0000 7ff61284e700 1 -- start start 2023-07-06T01:21:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.868+0000 7ff61284e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff60c0af2a0 0x7ff60c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.868+0000 7ff61284e700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff60c0cc320 0x7ff60c06a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff61284e700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60c0af670 con 0x7ff60c06a000 2023-07-06T01:21:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff61284e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60c06a9e0 con 0x7ff60c0af2a0 2023-07-06T01:21:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff61284e700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60c0cd560 con 0x7ff60c0cc320 2023-07-06T01:21:45.872 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff60bfff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff60c0af2a0 0x7ff60c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff60b7fe700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7ff60c06a000 0x7ff60c0cbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38828/0 (socket says 172.21.15.22:38828) 2023-07-06T01:21:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff60b7fe700 1 -- 172.21.15.22:0/1938568493 learned_addr learned my addr 172.21.15.22:0/1938568493 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff610deb700 1 --2- 172.21.15.22:0/1938568493 >> v2:172.21.15.103:3300/0 conn(0x7ff60c0cc320 0x7ff60c06a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff60bfff700 1 -- 172.21.15.22:0/1938568493 >> v2:172.21.15.103:3300/0 conn(0x7ff60c0cc320 msgr2=0x7ff60c06a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff60bfff700 1 --2- 172.21.15.22:0/1938568493 >> v2:172.21.15.103:3300/0 conn(0x7ff60c0cc320 0x7ff60c06a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff60bfff700 1 -- 172.21.15.22:0/1938568493 >> v1:172.21.15.22:6789/0 conn(0x7ff60c06a000 legacy=0x7ff60c0cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:45.873 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff60bfff700 1 -- 172.21.15.22:0/1938568493 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c06d030 con 0x7ff60c0af2a0 2023-07-06T01:21:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff60affd700 1 -- 172.21.15.22:0/1938568493 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112374684 0 0) 0x7ff60c0af670 con 0x7ff60c06a000 2023-07-06T01:21:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff610deb700 1 --2- 172.21.15.22:0/1938568493 >> v2:172.21.15.103:3300/0 conn(0x7ff60c0cc320 0x7ff60c06a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff60bfff700 1 --2- 172.21.15.22:0/1938568493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60c0af2a0 0x7ff60c06caf0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff60c160240 tx=0x7ff5fc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73809723308457d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff60affd700 1 -- 172.21.15.22:0/1938568493 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc011830 con 0x7ff60c0af2a0 2023-07-06T01:21:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff60affd700 1 -- 172.21.15.22:0/1938568493 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc011990 con 0x7ff60c0af2a0 2023-07-06T01:21:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.869+0000 7ff60affd700 1 -- 172.21.15.22:0/1938568493 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc011d20 con 0x7ff60c0af2a0 2023-07-06T01:21:45.874 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.870+0000 7ff61284e700 1 -- 172.21.15.22:0/1938568493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60c0af2a0 msgr2=0x7ff60c06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:45.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.870+0000 7ff61284e700 1 --2- 172.21.15.22:0/1938568493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60c0af2a0 0x7ff60c06caf0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff60c160240 tx=0x7ff5fc009b20 comp rx=0 tx=0).stop 2023-07-06T01:21:45.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.870+0000 7ff61284e700 1 -- 172.21.15.22:0/1938568493 shutdown_connections 2023-07-06T01:21:45.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.870+0000 7ff61284e700 1 --2- 172.21.15.22:0/1938568493 >> v2:172.21.15.103:3300/0 conn(0x7ff60c0cc320 0x7ff60c06a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:45.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.870+0000 7ff61284e700 1 --2- 172.21.15.22:0/1938568493 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60c0af2a0 0x7ff60c06caf0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:45.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.870+0000 7ff61284e700 1 -- 172.21.15.22:0/1938568493 >> 172.21.15.22:0/1938568493 conn(0x7ff60c0b1890 msgr2=0x7ff60c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:45.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.870+0000 7ff61284e700 1 -- 172.21.15.22:0/1938568493 shutdown_connections 2023-07-06T01:21:45.875 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.870+0000 7ff61284e700 1 -- 172.21.15.22:0/1938568493 wait complete. 2023-07-06T01:21:45.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.871+0000 7ff61284e700 1 Processor -- start 2023-07-06T01:21:45.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.871+0000 7ff61284e700 1 -- start start 2023-07-06T01:21:45.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff61284e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff60c06a000 0x7ff60c1f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:45.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff61284e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c0af2a0 0x7ff60c0b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:45.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff61284e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60c0cc320 0x7ff60c15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:45.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff61284e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c1f66d0 con 0x7ff60c0cc320 2023-07-06T01:21:45.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff61284e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0b4d90 con 0x7ff60c0af2a0 2023-07-06T01:21:45.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff61284e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c15f300 con 0x7ff60c06a000 2023-07-06T01:21:45.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff610deb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60c0cc320 0x7ff60c15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:45.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff610deb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60c0cc320 0x7ff60c15ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48094/0 (socket says 172.21.15.22:48094) 2023-07-06T01:21:45.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff610deb700 1 -- 172.21.15.22:0/2613000308 learned_addr learned my addr 172.21.15.22:0/2613000308 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:45.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff60b7fe700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c0af2a0 0x7ff60c0b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:45.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff60bfff700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff60c06a000 0x7ff60c1f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:45.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff610deb700 1 -- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff60c06a000 msgr2=0x7ff60c1f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:45.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff610deb700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff60c06a000 0x7ff60c1f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:45.879 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff610deb700 1 -- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c0af2a0 msgr2=0x7ff60c0b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:45.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.872+0000 7ff610deb700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c0af2a0 0x7ff60c0b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:45.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.873+0000 7ff610deb700 1 -- 172.21.15.22:0/2613000308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc0094a0 con 0x7ff60c0cc320 2023-07-06T01:21:45.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.873+0000 7ff610deb700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60c0cc320 0x7ff60c15ccc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff60400cd50 tx=0x7ff60400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:45.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.873+0000 7ff608ff9700 1 -- 172.21.15.22:0/2613000308 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60400f480 con 0x7ff60c0cc320 2023-07-06T01:21:45.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.873+0000 7ff608ff9700 1 -- 172.21.15.22:0/2613000308 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60400f600 con 0x7ff60c0cc320 2023-07-06T01:21:45.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.873+0000 7ff61284e700 1 -- 172.21.15.22:0/2613000308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60c15e6f0 con 0x7ff60c0cc320 2023-07-06T01:21:45.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.873+0000 7ff608ff9700 1 -- 172.21.15.22:0/2613000308 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60400f9b0 con 0x7ff60c0cc320 2023-07-06T01:21:45.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.873+0000 7ff61284e700 1 -- 172.21.15.22:0/2613000308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c15ecb0 con 0x7ff60c0cc320 2023-07-06T01:21:45.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.874+0000 7ff61284e700 1 -- 172.21.15.22:0/2613000308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60c15f4b0 con 0x7ff60c0cc320 2023-07-06T01:21:45.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.874+0000 7ff608ff9700 1 -- 172.21.15.22:0/2613000308 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff60400b040 con 0x7ff60c0cc320 2023-07-06T01:21:45.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.875+0000 7ff608ff9700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff5f806ace0 0x7ff5f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:45.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.875+0000 7ff608ff9700 1 -- 172.21.15.22:0/2613000308 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5739+0+0 (secure 0 0 0) 0x7ff604089a30 con 0x7ff60c0cc320 2023-07-06T01:21:45.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.875+0000 7ff60bfff700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff5f806ace0 0x7ff5f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:45.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.876+0000 7ff60bfff700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff5f806ace0 0x7ff5f806d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0090b0 tx=0x7ff5fc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:45.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:45.879+0000 7ff608ff9700 1 -- 172.21.15.22:0/2613000308 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff60c15f4b0 con 0x7ff60c0cc320 2023-07-06T01:21:46.081 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.079+0000 7ff61284e700 1 -- 172.21.15.22:0/2613000308 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff60c15f470 con 0x7ff60c0cc320 2023-07-06T01:21:46.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.716+0000 7ff608ff9700 1 -- 172.21.15.22:0/2613000308 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff604054cb0 con 0x7ff60c0cc320 2023-07-06T01:21:46.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.721+0000 7ff61284e700 1 -- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff5f806ace0 msgr2=0x7ff5f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:46.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.721+0000 7ff61284e700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff5f806ace0 0x7ff5f806d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0090b0 tx=0x7ff5fc008030 comp rx=0 tx=0).stop 2023-07-06T01:21:46.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.721+0000 7ff61284e700 1 -- 172.21.15.22:0/2613000308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60c0cc320 msgr2=0x7ff60c15ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:46.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.721+0000 7ff61284e700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60c0cc320 0x7ff60c15ccc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff60400cd50 tx=0x7ff60400a560 comp rx=0 tx=0).stop 2023-07-06T01:21:46.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.721+0000 7ff61284e700 1 -- 172.21.15.22:0/2613000308 shutdown_connections 2023-07-06T01:21:46.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.721+0000 7ff61284e700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff60c0cc320 0x7ff60c15ccc0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:46.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.721+0000 7ff61284e700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff60c06a000 0x7ff60c1f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:46.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.721+0000 7ff61284e700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff5f806ace0 0x7ff5f806d190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:46.725 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.721+0000 7ff61284e700 1 --2- 172.21.15.22:0/2613000308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff60c0af2a0 0x7ff60c0b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:46.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.721+0000 7ff61284e700 1 -- 172.21.15.22:0/2613000308 >> 172.21.15.22:0/2613000308 conn(0x7ff60c0b1890 msgr2=0x7ff60c0ada90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:46.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.721+0000 7ff61284e700 1 -- 172.21.15.22:0/2613000308 shutdown_connections 2023-07-06T01:21:46.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.722+0000 7ff61284e700 1 -- 172.21.15.22:0/2613000308 wait complete. 2023-07-06T01:21:46.727 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2023-07-06T01:21:46.741 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-06T01:21:46.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.888+0000 7f2d4d876700 1 Processor -- start 2023-07-06T01:21:46.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.888+0000 7f2d4d876700 1 -- start start 2023-07-06T01:21:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d4d876700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f2d480b36f0 0x7f2d480cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:46.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d4d876700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2d480cf5a0 0x7f2d480d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d4d876700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2d480b3b70 0x7f2d480d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d4d876700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d4815cff0 con 0x7f2d480b3b70 2023-07-06T01:21:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d4d876700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d480cebd0 con 0x7f2d480b36f0 2023-07-06T01:21:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d4d876700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d480cdfc0 con 0x7f2d480cf5a0 2023-07-06T01:21:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d477fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2d480b3b70 0x7f2d480d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d477fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f2d480b3b70 0x7f2d480d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48102/0 (socket says 172.21.15.22:48102) 2023-07-06T01:21:46.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d477fe700 1 -- 172.21.15.22:0/1847480979 learned_addr learned my addr 172.21.15.22:0/1847480979 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d467fc700 1 --2- 172.21.15.22:0/1847480979 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2d480cf5a0 0x7f2d480d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d46ffd700 1 --2- 172.21.15.22:0/1847480979 >> v2:172.21.15.103:3300/0 conn(0x7f2d480b36f0 0x7f2d480cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d477fe700 1 -- 172.21.15.22:0/1847480979 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2d480cf5a0 msgr2=0x7f2d480d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d477fe700 1 --2- 172.21.15.22:0/1847480979 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2d480cf5a0 0x7f2d480d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d477fe700 1 -- 172.21.15.22:0/1847480979 >> v2:172.21.15.103:3300/0 conn(0x7f2d480b36f0 msgr2=0x7f2d480cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d477fe700 1 --2- 172.21.15.22:0/1847480979 >> v2:172.21.15.103:3300/0 conn(0x7f2d480b36f0 0x7f2d480cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:46.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d477fe700 1 -- 172.21.15.22:0/1847480979 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d480ce2a0 con 0x7f2d480b3b70 2023-07-06T01:21:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.889+0000 7f2d477fe700 1 --2- 172.21.15.22:0/1847480979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d480b3b70 0x7f2d480d0fd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2d4808b8d0 tx=0x7f2d4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5663274fb79851 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.890+0000 7f2d45ffb700 1 -- 172.21.15.22:0/1847480979 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4000e070 con 0x7f2d480b3b70 2023-07-06T01:21:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.890+0000 7f2d45ffb700 1 -- 172.21.15.22:0/1847480979 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4000b910 con 0x7f2d480b3b70 2023-07-06T01:21:46.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.890+0000 7f2d45ffb700 1 -- 172.21.15.22:0/1847480979 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4000bcc0 con 0x7f2d480b3b70 2023-07-06T01:21:46.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.890+0000 7f2d4d876700 1 -- 172.21.15.22:0/1847480979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d480b3b70 msgr2=0x7f2d480d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.890+0000 7f2d4d876700 1 --2- 172.21.15.22:0/1847480979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d480b3b70 0x7f2d480d0fd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2d4808b8d0 tx=0x7f2d4000b330 comp rx=0 tx=0).stop 2023-07-06T01:21:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.890+0000 7f2d4d876700 1 -- 172.21.15.22:0/1847480979 shutdown_connections 2023-07-06T01:21:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.890+0000 7f2d4d876700 1 --2- 172.21.15.22:0/1847480979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d480b3b70 0x7f2d480d0fd0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.890+0000 7f2d4d876700 1 --2- 172.21.15.22:0/1847480979 >> v2:172.21.15.103:3300/0 conn(0x7f2d480b36f0 0x7f2d480cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:46.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.890+0000 7f2d4d876700 1 --2- 172.21.15.22:0/1847480979 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2d480cf5a0 0x7f2d480d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.890+0000 7f2d4d876700 1 -- 172.21.15.22:0/1847480979 >> 172.21.15.22:0/1847480979 conn(0x7f2d4809c2e0 msgr2=0x7f2d480d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.890+0000 7f2d4d876700 1 -- 172.21.15.22:0/1847480979 shutdown_connections 2023-07-06T01:21:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.890+0000 7f2d4d876700 1 -- 172.21.15.22:0/1847480979 wait complete. 2023-07-06T01:21:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.891+0000 7f2d4d876700 1 Processor -- start 2023-07-06T01:21:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.892+0000 7f2d4d876700 1 -- start start 2023-07-06T01:21:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.892+0000 7f2d4d876700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d480b36f0 0x7f2d480cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:46.899 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.892+0000 7f2d4d876700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2d480b3b70 0x7f2d4815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.892+0000 7f2d4d876700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d480cf5a0 0x7f2d480cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.892+0000 7f2d4d876700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d480ca7f0 con 0x7f2d480cf5a0 2023-07-06T01:21:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.892+0000 7f2d4d876700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d480cc180 con 0x7f2d480b36f0 2023-07-06T01:21:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.892+0000 7f2d4d876700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d480ca130 con 0x7f2d480b3b70 2023-07-06T01:21:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d477fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d480cf5a0 0x7f2d480cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d477fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d480cf5a0 0x7f2d480cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48108/0 (socket says 172.21.15.22:48108) 2023-07-06T01:21:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d477fe700 1 -- 172.21.15.22:0/1340006512 learned_addr learned my addr 172.21.15.22:0/1340006512 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:46.900 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d46ffd700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d480b36f0 0x7f2d480cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d467fc700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2d480b3b70 0x7f2d4815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d477fe700 1 -- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2d480b3b70 msgr2=0x7f2d4815a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d477fe700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2d480b3b70 0x7f2d4815a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d477fe700 1 -- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d480b36f0 msgr2=0x7f2d480cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d477fe700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d480b36f0 0x7f2d480cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d477fe700 1 -- 172.21.15.22:0/1340006512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d40009b30 con 0x7f2d480cf5a0 2023-07-06T01:21:46.901 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d477fe700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d480cf5a0 0x7f2d480cc910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2d48068130 tx=0x7f2d40004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:46.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d2ffff700 1 -- 172.21.15.22:0/1340006512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d400099b0 con 0x7f2d480cf5a0 2023-07-06T01:21:46.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d2ffff700 1 -- 172.21.15.22:0/1340006512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4000ce80 con 0x7f2d480cf5a0 2023-07-06T01:21:46.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d2ffff700 1 -- 172.21.15.22:0/1340006512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40004490 con 0x7f2d480cf5a0 2023-07-06T01:21:46.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d4d876700 1 -- 172.21.15.22:0/1340006512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d480ca410 con 0x7f2d480cf5a0 2023-07-06T01:21:46.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.893+0000 7f2d4d876700 1 -- 172.21.15.22:0/1340006512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d480cac10 con 0x7f2d480cf5a0 2023-07-06T01:21:46.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.894+0000 7f2d4d876700 1 -- 172.21.15.22:0/1340006512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d480b4840 con 0x7f2d480cf5a0 2023-07-06T01:21:46.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.895+0000 7f2d2ffff700 1 -- 172.21.15.22:0/1340006512 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2d400045f0 con 0x7f2d480cf5a0 2023-07-06T01:21:46.902 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.895+0000 7f2d2ffff700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2d3406ace0 0x7f2d3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.895+0000 7f2d2ffff700 1 -- 172.21.15.22:0/1340006512 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2d4008a3f0 con 0x7f2d480cf5a0 2023-07-06T01:21:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.895+0000 7f2d46ffd700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2d3406ace0 0x7f2d3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.896+0000 7f2d46ffd700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2d3406ace0 0x7f2d3406d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009ac0 tx=0x7f2d3c009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:46.903 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:46.900+0000 7f2d2ffff700 1 -- 172.21.15.22:0/1340006512 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2d40055830 con 0x7f2d480cf5a0 2023-07-06T01:21:47.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.105+0000 7f2d4d876700 1 -- 172.21.15.22:0/1340006512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2d480c9f90 con 0x7f2d480cf5a0 2023-07-06T01:21:47.110 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.108+0000 7f2d2ffff700 1 -- 172.21.15.22:0/1340006512 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2d480c9f90 con 0x7f2d480cf5a0 2023-07-06T01:21:47.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.110+0000 7f2d4d876700 1 -- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2d3406ace0 msgr2=0x7f2d3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:47.112 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.110+0000 7f2d4d876700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2d3406ace0 0x7f2d3406d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009ac0 tx=0x7f2d3c009210 comp rx=0 tx=0).stop 2023-07-06T01:21:47.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.110+0000 7f2d4d876700 1 -- 172.21.15.22:0/1340006512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d480cf5a0 msgr2=0x7f2d480cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:47.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.110+0000 7f2d4d876700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d480cf5a0 0x7f2d480cc910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2d48068130 tx=0x7f2d40004ad0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.110+0000 7f2d4d876700 1 -- 172.21.15.22:0/1340006512 shutdown_connections 2023-07-06T01:21:47.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.111+0000 7f2d4d876700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2d480cf5a0 0x7f2d480cc910 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.113 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.111+0000 7f2d4d876700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2d3406ace0 0x7f2d3406d190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.111+0000 7f2d4d876700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d480b36f0 0x7f2d480cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.111+0000 7f2d4d876700 1 --2- 172.21.15.22:0/1340006512 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2d480b3b70 0x7f2d4815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.111+0000 7f2d4d876700 1 -- 172.21.15.22:0/1340006512 >> 172.21.15.22:0/1340006512 conn(0x7f2d4809c2e0 msgr2=0x7f2d480d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:47.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.111+0000 7f2d4d876700 1 -- 172.21.15.22:0/1340006512 shutdown_connections 2023-07-06T01:21:47.114 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.111+0000 7f2d4d876700 1 -- 172.21.15.22:0/1340006512 wait complete. 2023-07-06T01:21:47.125 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2023-07-06T01:21:47.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.249+0000 7f143b26a700 1 Processor -- start 2023-07-06T01:21:47.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.249+0000 7f143b26a700 1 -- start start 2023-07-06T01:21:47.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.250+0000 7f143b26a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14340af2a0 0x7f143406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:47.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.250+0000 7f143b26a700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14340af670 con 0x7f143406a000 2023-07-06T01:21:47.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.250+0000 7f143b26a700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143406d030 con 0x7f14340cc320 2023-07-06T01:21:47.254 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.250+0000 7f143b26a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14340cc7a0 con 0x7f14340af2a0 2023-07-06T01:21:47.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.250+0000 7f1438805700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f143406a000 0x7f14340cbc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38860/0 (socket says 172.21.15.22:38860) 2023-07-06T01:21:47.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.250+0000 7f1438805700 1 -- 172.21.15.22:0/3216872272 learned_addr learned my addr 172.21.15.22:0/3216872272 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:47.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f1439006700 1 --2- 172.21.15.22:0/3216872272 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14340af2a0 0x7f143406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:47.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034718853 0 0) 0x7f14340af670 con 0x7f143406a000 2023-07-06T01:21:47.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14200035b0 con 0x7f143406a000 2023-07-06T01:21:47.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2698866153 0 0) 0x7f14200035b0 con 0x7f143406a000 2023-07-06T01:21:47.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14340af2a0 msgr2=0x7f143406caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:47.255 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 --2- 172.21.15.22:0/3216872272 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14340af2a0 0x7f143406caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 >> v1:172.21.15.103:6789/0 conn(0x7f14340cc320 legacy=0x7f1434160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:47.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14340cd560 con 0x7f143406a000 2023-07-06T01:21:47.256 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f1430001670 con 0x7f143406a000 2023-07-06T01:21:47.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 >> v1:172.21.15.22:6789/0 conn(0x7f143406a000 legacy=0x7f14340cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:47.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 --2- 172.21.15.22:0/3216872272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1420003a30 0x7f1420003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:47.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 --2- 172.21.15.22:0/3216872272 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14340af2a0 0x7f1420004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:47.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 --2- 172.21.15.22:0/3216872272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1420004d90 0x7f1420005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:47.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340cc7a0 con 0x7f1420003a30 2023-07-06T01:21:47.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142000bac0 con 0x7f1420004d90 2023-07-06T01:21:47.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142000bc50 con 0x7f14340af2a0 2023-07-06T01:21:47.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f1439006700 1 --2- 172.21.15.22:0/3216872272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1420003a30 0x7f1420003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:47.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f1438805700 1 --2- 172.21.15.22:0/3216872272 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14340af2a0 0x7f1420004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:47.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f1439807700 1 --2- 172.21.15.22:0/3216872272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1420004d90 0x7f1420005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:47.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f1439006700 1 -- 172.21.15.22:0/3216872272 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14340af2a0 msgr2=0x7f1420004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:47.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f1439006700 1 --2- 172.21.15.22:0/3216872272 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14340af2a0 0x7f1420004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f1439006700 1 -- 172.21.15.22:0/3216872272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1420004d90 msgr2=0x7f1420005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:47.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f1439006700 1 --2- 172.21.15.22:0/3216872272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1420004d90 0x7f1420005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.258 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f1439006700 1 -- 172.21.15.22:0/3216872272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14200035b0 con 0x7f1420003a30 2023-07-06T01:21:47.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.251+0000 7f1439807700 1 --2- 172.21.15.22:0/3216872272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1420004d90 0x7f1420005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:47.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.252+0000 7f1439006700 1 --2- 172.21.15.22:0/3216872272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1420003a30 0x7f1420003e40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1424009b60 tx=0x7f1420005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59c25466638a15b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:47.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.252+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142400e6c0 con 0x7f1420003a30 2023-07-06T01:21:47.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.252+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142400e820 con 0x7f1420003a30 2023-07-06T01:21:47.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.252+0000 7f142bfff700 1 -- 172.21.15.22:0/3216872272 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142400eb90 con 0x7f1420003a30 2023-07-06T01:21:47.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.252+0000 7f143b26a700 1 -- 172.21.15.22:0/3216872272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1420003a30 msgr2=0x7f1420003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:47.259 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.252+0000 7f143b26a700 1 --2- 172.21.15.22:0/3216872272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1420003a30 0x7f1420003e40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1424009b60 tx=0x7f1420005880 comp rx=0 tx=0).stop 2023-07-06T01:21:47.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.252+0000 7f1439807700 1 -- 172.21.15.22:0/3216872272 reap_dead start 2023-07-06T01:21:47.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.252+0000 7f143b26a700 1 -- 172.21.15.22:0/3216872272 shutdown_connections 2023-07-06T01:21:47.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.252+0000 7f143b26a700 1 -- 172.21.15.22:0/3216872272 >> 172.21.15.22:0/3216872272 conn(0x7f14340b1890 msgr2=0x7f14340b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:47.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.252+0000 7f143b26a700 1 -- 172.21.15.22:0/3216872272 shutdown_connections 2023-07-06T01:21:47.260 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.252+0000 7f143b26a700 1 -- 172.21.15.22:0/3216872272 wait complete. 2023-07-06T01:21:47.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.254+0000 7f143b26a700 1 Processor -- start 2023-07-06T01:21:47.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.254+0000 7f143b26a700 1 -- start start 2023-07-06T01:21:47.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.254+0000 7f143b26a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1420003a30 0x7f14340b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:47.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.254+0000 7f143b26a700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14340c9520 0x7f14341520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:47.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.254+0000 7f143b26a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14341f6020 0x7f143415eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:47.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.254+0000 7f143b26a700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340c9d80 con 0x7f14340c9520 2023-07-06T01:21:47.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.254+0000 7f143b26a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14341f5840 con 0x7f1420003a30 2023-07-06T01:21:47.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.254+0000 7f143b26a700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14340b4a40 con 0x7f14341f6020 2023-07-06T01:21:47.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.254+0000 7f1438805700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14340c9520 0x7f14341520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:47.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.254+0000 7f1438805700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14340c9520 0x7f14341520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48126/0 (socket says 172.21.15.22:48126) 2023-07-06T01:21:47.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.254+0000 7f1438805700 1 -- 172.21.15.22:0/2219677588 learned_addr learned my addr 172.21.15.22:0/2219677588 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:47.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.255+0000 7f1439006700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1420003a30 0x7f14340b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:47.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.255+0000 7f1439807700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14341f6020 0x7f143415eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:47.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.255+0000 7f1438805700 1 -- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14341f6020 msgr2=0x7f143415eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:47.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.255+0000 7f1438805700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14341f6020 0x7f143415eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.255+0000 7f1438805700 1 -- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1420003a30 msgr2=0x7f14340b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:47.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.255+0000 7f1438805700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1420003a30 0x7f14340b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.255+0000 7f1438805700 1 -- 172.21.15.22:0/2219677588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14340b4d20 con 0x7f14340c9520 2023-07-06T01:21:47.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.255+0000 7f1438805700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14340c9520 0x7f14341520f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f14300072c0 tx=0x7f1430007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:47.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.255+0000 7f1429ffb700 1 -- 172.21.15.22:0/2219677588 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143000f8d0 con 0x7f14340c9520 2023-07-06T01:21:47.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.255+0000 7f1429ffb700 1 -- 172.21.15.22:0/2219677588 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143000fa30 con 0x7f14340c9520 2023-07-06T01:21:47.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.255+0000 7f1429ffb700 1 -- 172.21.15.22:0/2219677588 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143000fde0 con 0x7f14340c9520 2023-07-06T01:21:47.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.255+0000 7f143b26a700 1 -- 172.21.15.22:0/2219677588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14340b5000 con 0x7f14340c9520 2023-07-06T01:21:47.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.255+0000 7f143b26a700 1 -- 172.21.15.22:0/2219677588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1434152630 con 0x7f14340c9520 2023-07-06T01:21:47.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.256+0000 7f143b26a700 1 -- 172.21.15.22:0/2219677588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14341f5d40 con 0x7f14340c9520 2023-07-06T01:21:47.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.257+0000 7f1429ffb700 1 -- 172.21.15.22:0/2219677588 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f143001d460 con 0x7f14340c9520 2023-07-06T01:21:47.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.257+0000 7f1429ffb700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f141806ad10 0x7f141806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:47.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.257+0000 7f1429ffb700 1 -- 172.21.15.22:0/2219677588 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1430089120 con 0x7f14340c9520 2023-07-06T01:21:47.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.257+0000 7f1439006700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f141806ad10 0x7f141806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:47.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.258+0000 7f1439006700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f141806ad10 0x7f141806d1c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f14240090b0 tx=0x7f1424008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:47.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.261+0000 7f1429ffb700 1 -- 172.21.15.22:0/2219677588 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f14341f5d40 con 0x7f14340c9520 2023-07-06T01:21:47.544 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.540+0000 7f143b26a700 1 -- 172.21.15.22:0/2219677588 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f14341f5d40 con 0x7f14340c9520 2023-07-06T01:21:47.717 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.716+0000 7f1429ffb700 1 -- 172.21.15.22:0/2219677588 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v136) v1 ==== 148+0+0 (secure 0 0 0) 0x7f14341f5d40 con 0x7f14340c9520 2023-07-06T01:21:47.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.718+0000 7f143b26a700 1 -- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f141806ad10 msgr2=0x7f141806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:47.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.718+0000 7f143b26a700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f141806ad10 0x7f141806d1c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f14240090b0 tx=0x7f1424008030 comp rx=0 tx=0).stop 2023-07-06T01:21:47.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.718+0000 7f143b26a700 1 -- 172.21.15.22:0/2219677588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14340c9520 msgr2=0x7f14341520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:47.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.718+0000 7f143b26a700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14340c9520 0x7f14341520f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f14300072c0 tx=0x7f1430007680 comp rx=0 tx=0).stop 2023-07-06T01:21:47.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.718+0000 7f143b26a700 1 -- 172.21.15.22:0/2219677588 shutdown_connections 2023-07-06T01:21:47.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.718+0000 7f143b26a700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f14341f6020 0x7f143415eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.718+0000 7f143b26a700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f141806ad10 0x7f141806d1c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.718+0000 7f143b26a700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1420003a30 0x7f14340b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.718+0000 7f143b26a700 1 --2- 172.21.15.22:0/2219677588 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14340c9520 0x7f14341520f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:47.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.718+0000 7f143b26a700 1 -- 172.21.15.22:0/2219677588 >> 172.21.15.22:0/2219677588 conn(0x7f14340b1890 msgr2=0x7f14340ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:47.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.718+0000 7f143b26a700 1 -- 172.21.15.22:0/2219677588 shutdown_connections 2023-07-06T01:21:47.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:47.718+0000 7f143b26a700 1 -- 172.21.15.22:0/2219677588 wait complete. 2023-07-06T01:21:47.732 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ERROR 2023-07-06T01:21:47.733 INFO:tasks.cephfs_test_runner: 2023-07-06T01:21:47.733 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-06T01:21:47.734 INFO:tasks.cephfs_test_runner:ERROR: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-07-06T01:21:47.734 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-06T01:21:47.734 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-06T01:21:47.734 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/test_scrub_checks.py", line 376, in test_flag_scrub_mdsdir 2023-07-06T01:21:47.734 INFO:tasks.cephfs_test_runner: self.scrub_with_stray_evaluation(self.fs, self.mount_a, "/", 2023-07-06T01:21:47.735 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/test_scrub_checks.py", line 323, in scrub_with_stray_evaluation 2023-07-06T01:21:47.735 INFO:tasks.cephfs_test_runner: fs.wait_until_scrub_complete(tag=out_json["scrub_tag"]), True) 2023-07-06T01:21:47.735 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/filesystem.py", line 1694, in wait_until_scrub_complete 2023-07-06T01:21:47.735 INFO:tasks.cephfs_test_runner: status = out_json['scrubs'][tag] 2023-07-06T01:21:47.735 INFO:tasks.cephfs_test_runner:KeyError: '3df72095-535c-4952-aa75-8713e98e5179' 2023-07-06T01:21:47.736 INFO:tasks.cephfs_test_runner: 2023-07-06T01:21:47.736 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-06T01:21:47.736 INFO:tasks.cephfs_test_runner:Ran 1 test in 110.758s 2023-07-06T01:21:47.736 INFO:tasks.cephfs_test_runner: 2023-07-06T01:21:47.736 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-07-06T01:21:47.737 INFO:tasks.cephfs_test_runner: 2023-07-06T01:21:47.737 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-06T01:21:47.737 INFO:tasks.cephfs_test_runner:ERROR: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-07-06T01:21:47.737 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-06T01:21:47.737 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-06T01:21:47.738 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/test_scrub_checks.py", line 376, in test_flag_scrub_mdsdir 2023-07-06T01:21:47.738 INFO:tasks.cephfs_test_runner: self.scrub_with_stray_evaluation(self.fs, self.mount_a, "/", 2023-07-06T01:21:47.738 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/test_scrub_checks.py", line 323, in scrub_with_stray_evaluation 2023-07-06T01:21:47.738 INFO:tasks.cephfs_test_runner: fs.wait_until_scrub_complete(tag=out_json["scrub_tag"]), True) 2023-07-06T01:21:47.738 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs/filesystem.py", line 1694, in wait_until_scrub_complete 2023-07-06T01:21:47.738 INFO:tasks.cephfs_test_runner: status = out_json['scrubs'][tag] 2023-07-06T01:21:47.739 INFO:tasks.cephfs_test_runner:KeyError: '3df72095-535c-4952-aa75-8713e98e5179' 2023-07-06T01:21:47.739 INFO:tasks.cephfs_test_runner: 2023-07-06T01:21:47.739 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-07-06T01:21:47.809 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=53471a5faa414ff7a03a682b498ee615 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_79365bb3b803780276d69efb2cc4afe5b160de9e/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2023-07-06T01:21:47.811 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-07-06T01:21:47.826 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-06T01:21:47.835 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-06T01:21:47.836 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:21:47.836 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:21:47.867 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19513 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T01:21:47.868 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2023-07-06T01:21:47.868 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:21:47.924 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19513 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T01:21:47.925 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:21:47.925 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:21:47.946 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284276k,nr_inodes=4071069,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19160 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T01:21:47.946 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2023-07-06T01:21:47.946 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T01:21:48.005 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284276k,nr_inodes=4071069,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19160 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T01:21:48.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:48.006 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-07-06T01:21:48.023 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T01:21:48.024 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T01:21:48.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:48.025 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T01:21:48.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:48.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T01:21:48.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:48.192 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2023-07-06T01:21:48.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:48.308 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2023-07-06T01:21:48.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:48.327 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2023-07-06T01:21:48.397 INFO:teuthology.orchestra.run.smithi022.stderr:Cannot find device "ceph-brx" 2023-07-06T01:21:48.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:21:48.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:48.399 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2023-07-06T01:21:48.420 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T01:21:48.420 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T01:21:48.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:48.421 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T01:21:48.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:48.505 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T01:21:48.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:48.586 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2023-07-06T01:21:48.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:48.740 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2023-07-06T01:21:48.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T01:21:48.758 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2023-07-06T01:21:48.827 INFO:teuthology.orchestra.run.smithi103.stderr:Cannot find device "ceph-brx" 2023-07-06T01:21:48.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:21:48.828 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-07-06T01:21:48.839 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-06T01:21:48.839 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:21:48.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.970+0000 7f31fe16e700 1 Processor -- start 2023-07-06T01:21:48.972 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.970+0000 7f31fe16e700 1 -- start start 2023-07-06T01:21:48.974 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.971+0000 7f31fe16e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31f815c3a0 0x7f31f80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:48.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.971+0000 7f31fe16e700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f31f80cd760 0x7f31f815d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:48.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.971+0000 7f31fe16e700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f806b9e0 con 0x7f31f806f910 2023-07-06T01:21:48.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.971+0000 7f31fe16e700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f80cb4a0 con 0x7f31f80cd760 2023-07-06T01:21:48.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.971+0000 7f31fe16e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f815d890 con 0x7f31f815c3a0 2023-07-06T01:21:48.975 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.971+0000 7f31f7fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f31f80cd760 0x7f31f815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:48.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.971+0000 7f31f7fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f31f80cd760 0x7f31f815d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48136/0 (socket says 172.21.15.22:48136) 2023-07-06T01:21:48.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.971+0000 7f31f7fff700 1 -- 172.21.15.22:0/3767526464 learned_addr learned my addr 172.21.15.22:0/3767526464 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:48.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.972+0000 7f31f77fe700 1 --2- 172.21.15.22:0/3767526464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31f815c3a0 0x7f31f80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:48.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.972+0000 7f31f7fff700 1 -- 172.21.15.22:0/3767526464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31f815c3a0 msgr2=0x7f31f80caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:48.976 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.972+0000 7f31f7fff700 1 --2- 172.21.15.22:0/3767526464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31f815c3a0 0x7f31f80caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:48.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.972+0000 7f31f7fff700 1 -- 172.21.15.22:0/3767526464 >> v1:172.21.15.103:6789/0 conn(0x7f31f806f910 legacy=0x7f31f815ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:48.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.972+0000 7f31f7fff700 1 -- 172.21.15.22:0/3767526464 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f806bc80 con 0x7f31f80cd760 2023-07-06T01:21:48.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.972+0000 7f31f7fff700 1 --2- 172.21.15.22:0/3767526464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f80cd760 0x7f31f815d190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f31f000bb90 tx=0x7f31f000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d19e451c96d111 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:48.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.972+0000 7f31f67fc700 1 -- 172.21.15.22:0/3767526464 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f000dc00 con 0x7f31f80cd760 2023-07-06T01:21:48.977 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.972+0000 7f31f67fc700 1 -- 172.21.15.22:0/3767526464 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f000dd60 con 0x7f31f80cd760 2023-07-06T01:21:48.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.972+0000 7f31f67fc700 1 -- 172.21.15.22:0/3767526464 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0004270 con 0x7f31f80cd760 2023-07-06T01:21:48.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.973+0000 7f31fe16e700 1 -- 172.21.15.22:0/3767526464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f80cd760 msgr2=0x7f31f815d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:48.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.973+0000 7f31fe16e700 1 --2- 172.21.15.22:0/3767526464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f80cd760 0x7f31f815d190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f31f000bb90 tx=0x7f31f000bea0 comp rx=0 tx=0).stop 2023-07-06T01:21:48.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.973+0000 7f31fe16e700 1 -- 172.21.15.22:0/3767526464 shutdown_connections 2023-07-06T01:21:48.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.973+0000 7f31fe16e700 1 --2- 172.21.15.22:0/3767526464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f80cd760 0x7f31f815d190 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:48.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.973+0000 7f31fe16e700 1 --2- 172.21.15.22:0/3767526464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31f815c3a0 0x7f31f80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:48.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.973+0000 7f31fe16e700 1 -- 172.21.15.22:0/3767526464 >> 172.21.15.22:0/3767526464 conn(0x7f31f80b0710 msgr2=0x7f31f80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:48.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.973+0000 7f31fe16e700 1 -- 172.21.15.22:0/3767526464 shutdown_connections 2023-07-06T01:21:48.979 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.973+0000 7f31fe16e700 1 -- 172.21.15.22:0/3767526464 wait complete. 2023-07-06T01:21:48.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.974+0000 7f31fe16e700 1 Processor -- start 2023-07-06T01:21:48.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.974+0000 7f31fe16e700 1 -- start start 2023-07-06T01:21:48.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31fe16e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31f806f910 0x7f31f8150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:48.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31fe16e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31f80cd760 0x7f31f80b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:48.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31fe16e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f815c3a0 0x7f31f80b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:48.982 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31fe16e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f815b010 con 0x7f31f815c3a0 2023-07-06T01:21:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31fe16e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f81590b0 con 0x7f31f806f910 2023-07-06T01:21:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31fe16e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f8150850 con 0x7f31f80cd760 2023-07-06T01:21:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31f7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f815c3a0 0x7f31f80b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31f7fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f815c3a0 0x7f31f80b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48142/0 (socket says 172.21.15.22:48142) 2023-07-06T01:21:48.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31f7fff700 1 -- 172.21.15.22:0/3621973762 learned_addr learned my addr 172.21.15.22:0/3621973762 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:48.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31f6ffd700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31f80cd760 0x7f31f80b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:48.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31f77fe700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31f806f910 0x7f31f8150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:48.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31f7fff700 1 -- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31f80cd760 msgr2=0x7f31f80b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:48.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31f7fff700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31f80cd760 0x7f31f80b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:48.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31f7fff700 1 -- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31f806f910 msgr2=0x7f31f8150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:48.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31f7fff700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31f806f910 0x7f31f8150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:48.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31f7fff700 1 -- 172.21.15.22:0/3621973762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f000b7f0 con 0x7f31f815c3a0 2023-07-06T01:21:48.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31f6ffd700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31f80cd760 0x7f31f80b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:48.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31f77fe700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31f806f910 0x7f31f8150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:48.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.975+0000 7f31f7fff700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f815c3a0 0x7f31f80b1a10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f31f000d6d0 tx=0x7f31f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:48.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.976+0000 7f31effff700 1 -- 172.21.15.22:0/3621973762 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0003bc0 con 0x7f31f815c3a0 2023-07-06T01:21:48.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.976+0000 7f31effff700 1 -- 172.21.15.22:0/3621973762 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f0003d20 con 0x7f31f815c3a0 2023-07-06T01:21:48.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.976+0000 7f31effff700 1 -- 172.21.15.22:0/3621973762 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f00226b0 con 0x7f31f815c3a0 2023-07-06T01:21:48.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.976+0000 7f31fe16e700 1 -- 172.21.15.22:0/3621973762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f8159f70 con 0x7f31f815c3a0 2023-07-06T01:21:48.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.976+0000 7f31fe16e700 1 -- 172.21.15.22:0/3621973762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f815a530 con 0x7f31f815c3a0 2023-07-06T01:21:48.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.977+0000 7f31fe16e700 1 -- 172.21.15.22:0/3621973762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f806fd60 con 0x7f31f815c3a0 2023-07-06T01:21:48.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.977+0000 7f31effff700 1 -- 172.21.15.22:0/3621973762 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f31f0015070 con 0x7f31f815c3a0 2023-07-06T01:21:48.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.977+0000 7f31effff700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f31d806ace0 0x7f31d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:48.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.978+0000 7f31effff700 1 -- 172.21.15.22:0/3621973762 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f31f008cd90 con 0x7f31f815c3a0 2023-07-06T01:21:48.987 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.978+0000 7f31f77fe700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f31d806ace0 0x7f31d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:48.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.978+0000 7f31f77fe700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f31d806ace0 0x7f31d806d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009f50 tx=0x7f31e800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:48.988 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:48.982+0000 7f31effff700 1 -- 172.21.15.22:0/3621973762 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f31f806fd60 con 0x7f31f815c3a0 2023-07-06T01:21:49.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.175+0000 7f31fe16e700 1 -- 172.21.15.22:0/3621973762 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f31f806c240 con 0x7f31d806ace0 2023-07-06T01:21:49.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.178+0000 7f31effff700 1 -- 172.21.15.22:0/3621973762 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73464 (secure 0 0 0) 0x7f31e8007c40 con 0x7f31d806ace0 2023-07-06T01:21:49.435 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:21:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.181+0000 7f31fe16e700 1 -- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f31d806ace0 msgr2=0x7f31d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.181+0000 7f31fe16e700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f31d806ace0 0x7f31d806d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009f50 tx=0x7f31e800b040 comp rx=0 tx=0).stop 2023-07-06T01:21:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.181+0000 7f31fe16e700 1 -- 172.21.15.22:0/3621973762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f815c3a0 msgr2=0x7f31f80b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:49.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.181+0000 7f31fe16e700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f815c3a0 0x7f31f80b1a10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f31f000d6d0 tx=0x7f31f0004750 comp rx=0 tx=0).stop 2023-07-06T01:21:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.181+0000 7f31fe16e700 1 -- 172.21.15.22:0/3621973762 shutdown_connections 2023-07-06T01:21:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.181+0000 7f31fe16e700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f31f815c3a0 0x7f31f80b1a10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.181+0000 7f31fe16e700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f31d806ace0 0x7f31d806d190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.181+0000 7f31fe16e700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31f806f910 0x7f31f8150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.181+0000 7f31fe16e700 1 --2- 172.21.15.22:0/3621973762 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f31f80cd760 0x7f31f80b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.181+0000 7f31fe16e700 1 -- 172.21.15.22:0/3621973762 >> 172.21.15.22:0/3621973762 conn(0x7f31f80b0710 msgr2=0x7f31f806c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.181+0000 7f31fe16e700 1 -- 172.21.15.22:0/3621973762 shutdown_connections 2023-07-06T01:21:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.181+0000 7f31fe16e700 1 -- 172.21.15.22:0/3621973762 wait complete. 2023-07-06T01:21:49.439 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:21:49.439 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-07-06T01:21:48.035189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":323796,"kb_used_data":14740,"kb_used_omap":0,"kb_used_meta":308992,"kb_avail":754650924,"statfs":{"total":773094113280,"available":772762546176,"internally_reserved":0,"allocated":15093760,"data_stored":12982602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316407808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14607696,"num_objects":-240,"num_object_clones":0,"num_object_copies":-480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-240,"num_whiteouts":0,"num_read":-2112,"num_read_kb":-2464,"num_write":-17492,"num_write_kb":-100777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-29,"num_bytes_recovered":-94,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001573"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-07-06T01:21:41.674722+0000","last_change":"2023-07-06T01:19:20.074195+0000","last_active":"2023-07-06T01:21:41.674722+0000","last_peered":"2023-07-06T01:21:41.674722+0000","last_clean":"2023-07-06T01:21:41.674722+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:21:41.674722+0000","last_undegraded":"2023-07-06T01:21:41.674722+0000","last_fullsized":"2023-07-06T01:21:41.674722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:57:44.636425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.754269+0000","last_change":"2023-07-06T01:19:19.587110+0000","last_active":"2023-07-06T01:21:46.754269+0000","last_peered":"2023-07-06T01:21:46.754269+0000","last_clean":"2023-07-06T01:21:46.754269+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:21:46.754269+0000","last_undegraded":"2023-07-06T01:21:46.754269+0000","last_fullsized":"2023-07-06T01:21:46.754269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:38.656516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":50,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.760757+0000","last_change":"2023-07-06T01:19:19.584242+0000","last_active":"2023-07-06T01:21:46.760757+0000","last_peered":"2023-07-06T01:21:46.760757+0000","last_clean":"2023-07-06T01:21:46.760757+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:21:46.760757+0000","last_undegraded":"2023-07-06T01:21:46.760757+0000","last_fullsized":"2023-07-06T01:21:46.760757+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:26:02.484123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":198,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-07-06T01:21:41.185424+0000","last_change":"2023-07-06T01:19:19.586495+0000","last_active":"2023-07-06T01:21:41.185424+0000","last_peered":"2023-07-06T01:21:41.185424+0000","last_clean":"2023-07-06T01:21:41.185424+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:21:41.185424+0000","last_undegraded":"2023-07-06T01:21:41.185424+0000","last_fullsized":"2023-07-06T01:21:41.185424+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:45:30.569963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-07-06T01:21:41.183904+0000","last_change":"2023-07-06T01:19:19.586426+0000","last_active":"2023-07-06T01:21:41.183904+0000","last_peered":"2023-07-06T01:21:41.183904+0000","last_clean":"2023-07-06T01:21:41.183904+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:21:41.183904+0000","last_undegraded":"2023-07-06T01:21:41.183904+0000","last_fullsized":"2023-07-06T01:21:41.183904+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:59:03.004424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-07-06T01:21:41.184889+0000","last_change":"2023-07-06T01:19:19.586626+0000","last_active":"2023-07-06T01:21:41.184889+0000","last_peered":"2023-07-06T01:21:41.184889+0000","last_clean":"2023-07-06T01:21:41.184889+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:21:41.184889+0000","last_undegraded":"2023-07-06T01:21:41.184889+0000","last_fullsized":"2023-07-06T01:21:41.184889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:52:50.952542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.756290+0000","last_change":"2023-07-06T01:19:19.584086+0000","last_active":"2023-07-06T01:21:46.756290+0000","last_peered":"2023-07-06T01:21:46.756290+0000","last_clean":"2023-07-06T01:21:46.756290+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:21:46.756290+0000","last_undegraded":"2023-07-06T01:21:46.756290+0000","last_fullsized":"2023-07-06T01:21:46.756290+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:03.031857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":50,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.725480+0000","last_change":"2023-07-06T01:19:19.586424+0000","last_active":"2023-07-06T01:21:46.725480+0000","last_peered":"2023-07-06T01:21:46.725480+0000","last_clean":"2023-07-06T01:21:46.725480+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:21:46.725480+0000","last_undegraded":"2023-07-06T01:21:46.725480+0000","last_fullsized":"2023-07-06T01:21:46.725480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:08:16.360359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":50,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.725266+0000","last_change":"2023-07-06T01:19:19.583699+0000","last_active":"2023-07-06T01:21:46.725266+0000","last_peered":"2023-07-06T01:21:46.725266+0000","last_clean":"2023-07-06T01:21:46.725266+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:21:46.725266+0000","last_undegraded":"2023-07-06T01:21:46.725266+0000","last_fullsized":"2023-07-06T01:21:46.725266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:16:36.301039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542177,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50108,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321732,"statfs":{"total":96636764160,"available":96585453568,"internally_reserved":0,"allocated":1036288,"data_stored":774949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46500000000000002,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.623,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.497},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48999999999999999,"15min":0.49199999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69699999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.55500000000000005,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.61499999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.42699999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72199999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.439},{"interface":"front","average":{"1min":0.505,"5min":0.58299999999999996,"15min":0.59599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69499999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.746},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.68300000000000005,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.66100000000000003,"15min":0.67700000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71899999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67700000000000005}]}]},{"osd":4,"up_from":14,"seq":60129542177,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60880,"kb_used_data":7688,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94310960,"statfs":{"total":96636764160,"available":96574423040,"internally_reserved":0,"allocated":7872512,"data_stored":7612965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68600000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.38900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68799999999999994,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.443}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.502},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.74299999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79400000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57499999999999996}]},{"osd":5,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.32800000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72199999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.68500000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79400000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70799999999999996}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64600000000000002,"15min":0.65800000000000003},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81200000000000006,"5min":0.872,"15min":0.872},"last":0.68600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36188,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335652,"statfs":{"total":96636764160,"available":96599707648,"internally_reserved":0,"allocated":1069056,"data_stored":792978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":1.2589999999999999,"15min":1.4139999999999999},"min":{"1min":0.23200000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.51300000000000001,"5min":12.23,"15min":12.23},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.312,"5min":1.5329999999999999,"15min":1.7370000000000001},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.375,"5min":15.208,"15min":15.208},"last":0.31900000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.89900000000000002,"15min":0.98499999999999999},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":1.5189999999999999,"5min":12.074,"15min":12.074},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":1.306,"15min":1.458},"min":{"1min":0.26500000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73999999999999999,"5min":11.988,"15min":11.988},"last":0.34799999999999998}]},{"osd":3,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":1.8120000000000001,"15min":2.052},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.47999999999999998,"5min":18.963000000000001,"15min":18.963000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":1.5800000000000001,"15min":1.7849999999999999},"min":{"1min":0.24399999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.41299999999999998,"5min":15.244999999999999,"15min":15.244999999999999},"last":0.373}]},{"osd":4,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":1.127,"15min":1.2010000000000001},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.6080000000000001,"5min":11.923,"15min":11.923},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":1.5249999999999999,"15min":1.651},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.5950000000000002,"5min":12.019,"15min":12.019},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":1.099,"15min":1.1850000000000001},"min":{"1min":0.40100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79700000000000004,"5min":11.999000000000001,"15min":11.999000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":1.5,"15min":1.6499999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88800000000000001,"5min":11.951000000000001,"15min":11.951000000000001},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":1.788,"15min":1.9690000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.7490000000000001,"5min":15.167999999999999,"15min":15.167999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":1.1479999999999999,"15min":1.236},"min":{"1min":0.47799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":12.041,"15min":12.041},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":1.8180000000000001,"15min":2.008},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.625,"5min":15.115,"15min":15.115},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":1.163,"15min":1.2410000000000001},"min":{"1min":0.45700000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8200000000000001,"5min":11.965,"15min":11.965},"last":0.63100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.35499999999999998,"15min":0.35999999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60199999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45400000000000001,"15min":0.45500000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66800000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0880000000000001,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.56200000000000006}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.45400000000000001,"15min":0.46400000000000002},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.88800000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.496},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.45700000000000002,"15min":0.46500000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91400000000000003,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.41499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.97199999999999998,"5min":2.097,"15min":2.097},"last":0.629},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.62},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.196,"5min":0.71699999999999997,"15min":0.63700000000000001},"min":{"1min":0.44,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":11.619,"5min":11.619,"15min":11.619},"last":0.59399999999999997}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.80600000000000005,"15min":0.73199999999999998},"min":{"1min":0.50800000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":11.603999999999999,"5min":11.603999999999999,"15min":11.603999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.93899999999999995,"15min":0.98399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.518,"5min":6.3049999999999997,"15min":6.3049999999999997},"last":0.86599999999999999}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.94799999999999995,"15min":0.99099999999999999},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.609,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.2430000000000001,"5min":0.74399999999999999,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":11.663,"5min":11.663,"15min":11.663},"last":0.66000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35592,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336248,"statfs":{"total":96636764160,"available":96600317952,"internally_reserved":0,"allocated":458752,"data_stored":196675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.25700000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.497},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.33900000000000002},"min":{"1min":0.28799999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.371,"15min":0.36699999999999999},"min":{"1min":0.27100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.34200000000000003,"15min":0.33200000000000002},"min":{"1min":0.28899999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.42399999999999999,"15min":0.42199999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61199999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.47899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.47099999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.625},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35592,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336248,"statfs":{"total":96636764160,"available":96600317952,"internally_reserved":0,"allocated":458752,"data_stored":196675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.45300000000000001,"15min":0.46999999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67800000000000005,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.46200000000000002,"15min":0.47799999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.52800000000000002,"5min":1.51,"15min":1.51},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.25700000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64800000000000002,"5min":1.331,"15min":1.331},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.53400000000000003,"15min":0.56000000000000005},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.58099999999999996,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.53700000000000003,"15min":0.56100000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53600000000000003,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.376},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.53700000000000003,"15min":0.56499999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.501,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.67500000000000004,"15min":0.69499999999999995},"min":{"1min":0.44700000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.71999999999999997,"5min":1.663,"15min":1.663},"last":0.621},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.66500000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71499999999999997,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.64200000000000002,"15min":0.65800000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70199999999999996,"5min":1.667,"15min":1.667},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.68400000000000005,"15min":0.69999999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77500000000000002,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.65000000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.65700000000000003,"15min":0.67400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78100000000000003,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.78800000000000003,"15min":0.82599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76500000000000001,"5min":3.8130000000000002,"15min":3.8130000000000002},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.72399999999999998,"15min":0.753},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72599999999999998,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.75900000000000001,"15min":0.78200000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92600000000000005,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.51200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34248,"kb_used_data":3200,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94337592,"statfs":{"total":96636764160,"available":96601694208,"internally_reserved":0,"allocated":3276800,"data_stored":3014907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.499},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51700000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61899999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.36199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63100000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.432},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.53300000000000003,"15min":0.54400000000000004},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.61699999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73099999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.59399999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81200000000000006,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.58299999999999996,"15min":0.59499999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.627}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.67600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35580,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336260,"statfs":{"total":96636764160,"available":96600330240,"internally_reserved":0,"allocated":446464,"data_stored":190843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.497},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.503,"15min":0.503},"min":{"1min":0.38200000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66700000000000004,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.33000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.64600000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.56200000000000006}]},{"osd":2,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67400000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.54100000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68600000000000005,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.66200000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54100000000000004,"15min":0.54600000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67800000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.63200000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.63300000000000001}]},{"osd":7,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71199999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.745},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.66000000000000003,"5min":0.87,"15min":0.87},"last":0.67900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:21:49.442 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:21:49.568 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.566+0000 7f1258c12700 1 Processor -- start 2023-07-06T01:21:49.569 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.566+0000 7f1258c12700 1 -- start start 2023-07-06T01:21:49.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.567+0000 7f1258c12700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f125415cb50 0x7f125406a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:49.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.567+0000 7f1258c12700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f12540cf260 0x7f1254160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:49.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.567+0000 7f1258c12700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12540ca2b0 con 0x7f125406a540 2023-07-06T01:21:49.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.567+0000 7f1258c12700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12540af440 con 0x7f12540cf260 2023-07-06T01:21:49.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.567+0000 7f1258c12700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125406a970 con 0x7f125415cb50 2023-07-06T01:21:49.572 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f1252d9d700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f12540cf260 0x7f1254160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f125259c700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f125415cb50 0x7f125406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:49.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f1252d9d700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f12540cf260 0x7f1254160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:36874/0 (socket says 172.21.15.22:36874) 2023-07-06T01:21:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f1251d9b700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f125406a540 0x7f12540ceb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:40954/0 (socket says 172.21.15.22:40954) 2023-07-06T01:21:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f1252d9d700 1 -- 172.21.15.22:0/2708915897 learned_addr learned my addr 172.21.15.22:0/2708915897 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f125259c700 1 -- 172.21.15.22:0/2708915897 >> v2:172.21.15.103:3300/0 conn(0x7f12540cf260 msgr2=0x7f1254160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f125259c700 1 --2- 172.21.15.22:0/2708915897 >> v2:172.21.15.103:3300/0 conn(0x7f12540cf260 0x7f1254160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f125259c700 1 -- 172.21.15.22:0/2708915897 >> v1:172.21.15.103:6789/0 conn(0x7f125406a540 legacy=0x7f12540ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f125159a700 1 -- 172.21.15.22:0/2708915897 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627044092 0 0) 0x7f12540ca2b0 con 0x7f125406a540 2023-07-06T01:21:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f125259c700 1 -- 172.21.15.22:0/2708915897 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12540cf6e0 con 0x7f125415cb50 2023-07-06T01:21:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f1252d9d700 1 --2- 172.21.15.22:0/2708915897 >> v2:172.21.15.103:3300/0 conn(0x7f12540cf260 0x7f1254160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f125259c700 1 --2- 172.21.15.22:0/2708915897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f125415cb50 0x7f125406a000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1244009840 tx=0x7f1244009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1db68c9680b60722 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:49.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f125159a700 1 -- 172.21.15.22:0/2708915897 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244011830 con 0x7f125415cb50 2023-07-06T01:21:49.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f125159a700 1 -- 172.21.15.22:0/2708915897 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1244011990 con 0x7f125415cb50 2023-07-06T01:21:49.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.568+0000 7f125159a700 1 -- 172.21.15.22:0/2708915897 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244011d20 con 0x7f125415cb50 2023-07-06T01:21:49.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.569+0000 7f1258c12700 1 -- 172.21.15.22:0/2708915897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f125415cb50 msgr2=0x7f125406a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:49.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.569+0000 7f1258c12700 1 --2- 172.21.15.22:0/2708915897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f125415cb50 0x7f125406a000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1244009840 tx=0x7f1244009b50 comp rx=0 tx=0).stop 2023-07-06T01:21:49.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.569+0000 7f1258c12700 1 -- 172.21.15.22:0/2708915897 shutdown_connections 2023-07-06T01:21:49.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.569+0000 7f1258c12700 1 --2- 172.21.15.22:0/2708915897 >> v2:172.21.15.103:3300/0 conn(0x7f12540cf260 0x7f1254160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.569+0000 7f1258c12700 1 --2- 172.21.15.22:0/2708915897 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f125415cb50 0x7f125406a000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.569+0000 7f1258c12700 1 -- 172.21.15.22:0/2708915897 >> 172.21.15.22:0/2708915897 conn(0x7f125406b110 msgr2=0x7f125406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:49.577 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.569+0000 7f1258c12700 1 -- 172.21.15.22:0/2708915897 shutdown_connections 2023-07-06T01:21:49.578 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.569+0000 7f1258c12700 1 -- 172.21.15.22:0/2708915897 wait complete. 2023-07-06T01:21:49.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.571+0000 7f1258c12700 1 Processor -- start 2023-07-06T01:21:49.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.571+0000 7f1258c12700 1 -- start start 2023-07-06T01:21:49.580 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1258c12700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f125406a540 0x7f12540af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1258c12700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12540cf260 0x7f12540b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1258c12700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f125415cb50 0x7f125414f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:49.581 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1258c12700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12540b0930 con 0x7f12540cf260 2023-07-06T01:21:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1258c12700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125414f630 con 0x7f125406a540 2023-07-06T01:21:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1258c12700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12540afcb0 con 0x7f125415cb50 2023-07-06T01:21:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1251d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12540cf260 0x7f12540b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1251d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12540cf260 0x7f12540b0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48154/0 (socket says 172.21.15.22:48154) 2023-07-06T01:21:49.582 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1251d9b700 1 -- 172.21.15.22:0/3940825304 learned_addr learned my addr 172.21.15.22:0/3940825304 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:49.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f125259c700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f125406a540 0x7f12540af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:49.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1252d9d700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f125415cb50 0x7f125414f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:49.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1251d9b700 1 -- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f125415cb50 msgr2=0x7f125414f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:49.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1251d9b700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f125415cb50 0x7f125414f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.583 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1251d9b700 1 -- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f125406a540 msgr2=0x7f12540af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:49.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1251d9b700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f125406a540 0x7f12540af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1251d9b700 1 -- 172.21.15.22:0/3940825304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c009b30 con 0x7f12540cf260 2023-07-06T01:21:49.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.572+0000 7f1251d9b700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12540cf260 0x7f12540b0d50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f123c0011e0 tx=0x7f123c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:49.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.573+0000 7f124affd700 1 -- 172.21.15.22:0/3940825304 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c004c80 con 0x7f12540cf260 2023-07-06T01:21:49.584 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.573+0000 7f124affd700 1 -- 172.21.15.22:0/3940825304 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c004e00 con 0x7f12540cf260 2023-07-06T01:21:49.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.573+0000 7f1258c12700 1 -- 172.21.15.22:0/3940825304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12440094a0 con 0x7f12540cf260 2023-07-06T01:21:49.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.573+0000 7f124affd700 1 -- 172.21.15.22:0/3940825304 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c007310 con 0x7f12540cf260 2023-07-06T01:21:49.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.573+0000 7f1258c12700 1 -- 172.21.15.22:0/3940825304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12540b0270 con 0x7f12540cf260 2023-07-06T01:21:49.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.574+0000 7f1258c12700 1 -- 172.21.15.22:0/3940825304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12540744b0 con 0x7f12540cf260 2023-07-06T01:21:49.585 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.574+0000 7f124affd700 1 -- 172.21.15.22:0/3940825304 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f123c007470 con 0x7f12540cf260 2023-07-06T01:21:49.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.574+0000 7f124affd700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f123406ace0 0x7f123406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:49.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.575+0000 7f124affd700 1 -- 172.21.15.22:0/3940825304 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f123c088da0 con 0x7f12540cf260 2023-07-06T01:21:49.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.575+0000 7f125259c700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f123406ace0 0x7f123406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:49.586 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.575+0000 7f125259c700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f123406ace0 0x7f123406d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1244000c00 tx=0x7f124401c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:49.587 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.579+0000 7f124affd700 1 -- 172.21.15.22:0/3940825304 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f123c054150 con 0x7f12540cf260 2023-07-06T01:21:49.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.770+0000 7f1258c12700 1 -- 172.21.15.22:0/3940825304 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f12540cff30 con 0x7f123406ace0 2023-07-06T01:21:49.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.774+0000 7f124affd700 1 -- 172.21.15.22:0/3940825304 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73464 (secure 0 0 0) 0x7f1244011b80 con 0x7f123406ace0 2023-07-06T01:21:49.777 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:21:49.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.776+0000 7f1258c12700 1 -- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f123406ace0 msgr2=0x7f123406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:49.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.776+0000 7f1258c12700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f123406ace0 0x7f123406d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1244000c00 tx=0x7f124401c040 comp rx=0 tx=0).stop 2023-07-06T01:21:49.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.776+0000 7f1258c12700 1 -- 172.21.15.22:0/3940825304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12540cf260 msgr2=0x7f12540b0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:49.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.776+0000 7f1258c12700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12540cf260 0x7f12540b0d50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f123c0011e0 tx=0x7f123c00ca30 comp rx=0 tx=0).stop 2023-07-06T01:21:49.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.776+0000 7f1258c12700 1 -- 172.21.15.22:0/3940825304 shutdown_connections 2023-07-06T01:21:49.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.776+0000 7f1258c12700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f125415cb50 0x7f125414f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.776+0000 7f1258c12700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f123406ace0 0x7f123406d190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.776+0000 7f1258c12700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f125406a540 0x7f12540af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.779 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.776+0000 7f1258c12700 1 --2- 172.21.15.22:0/3940825304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12540cf260 0x7f12540b0d50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.776+0000 7f1258c12700 1 -- 172.21.15.22:0/3940825304 >> 172.21.15.22:0/3940825304 conn(0x7f125406b110 msgr2=0x7f12540ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:49.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.776+0000 7f1258c12700 1 -- 172.21.15.22:0/3940825304 shutdown_connections 2023-07-06T01:21:49.780 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.776+0000 7f1258c12700 1 -- 172.21.15.22:0/3940825304 wait complete. 2023-07-06T01:21:49.780 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:21:49.790 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-07-06T01:21:48.035189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":323796,"kb_used_data":14740,"kb_used_omap":0,"kb_used_meta":308992,"kb_avail":754650924,"statfs":{"total":773094113280,"available":772762546176,"internally_reserved":0,"allocated":15093760,"data_stored":12982602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316407808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14607696,"num_objects":-240,"num_object_clones":0,"num_object_copies":-480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-240,"num_whiteouts":0,"num_read":-2112,"num_read_kb":-2464,"num_write":-17492,"num_write_kb":-100777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-29,"num_bytes_recovered":-94,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001573"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-07-06T01:21:41.674722+0000","last_change":"2023-07-06T01:19:20.074195+0000","last_active":"2023-07-06T01:21:41.674722+0000","last_peered":"2023-07-06T01:21:41.674722+0000","last_clean":"2023-07-06T01:21:41.674722+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:21:41.674722+0000","last_undegraded":"2023-07-06T01:21:41.674722+0000","last_fullsized":"2023-07-06T01:21:41.674722+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:57:44.636425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.754269+0000","last_change":"2023-07-06T01:19:19.587110+0000","last_active":"2023-07-06T01:21:46.754269+0000","last_peered":"2023-07-06T01:21:46.754269+0000","last_clean":"2023-07-06T01:21:46.754269+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:21:46.754269+0000","last_undegraded":"2023-07-06T01:21:46.754269+0000","last_fullsized":"2023-07-06T01:21:46.754269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:38.656516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":50,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.760757+0000","last_change":"2023-07-06T01:19:19.584242+0000","last_active":"2023-07-06T01:21:46.760757+0000","last_peered":"2023-07-06T01:21:46.760757+0000","last_clean":"2023-07-06T01:21:46.760757+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:21:46.760757+0000","last_undegraded":"2023-07-06T01:21:46.760757+0000","last_fullsized":"2023-07-06T01:21:46.760757+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:26:02.484123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":198,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-07-06T01:21:41.185424+0000","last_change":"2023-07-06T01:19:19.586495+0000","last_active":"2023-07-06T01:21:41.185424+0000","last_peered":"2023-07-06T01:21:41.185424+0000","last_clean":"2023-07-06T01:21:41.185424+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:21:41.185424+0000","last_undegraded":"2023-07-06T01:21:41.185424+0000","last_fullsized":"2023-07-06T01:21:41.185424+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:45:30.569963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-07-06T01:21:41.183904+0000","last_change":"2023-07-06T01:19:19.586426+0000","last_active":"2023-07-06T01:21:41.183904+0000","last_peered":"2023-07-06T01:21:41.183904+0000","last_clean":"2023-07-06T01:21:41.183904+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:21:41.183904+0000","last_undegraded":"2023-07-06T01:21:41.183904+0000","last_fullsized":"2023-07-06T01:21:41.183904+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:59:03.004424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-07-06T01:21:41.184889+0000","last_change":"2023-07-06T01:19:19.586626+0000","last_active":"2023-07-06T01:21:41.184889+0000","last_peered":"2023-07-06T01:21:41.184889+0000","last_clean":"2023-07-06T01:21:41.184889+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:21:41.184889+0000","last_undegraded":"2023-07-06T01:21:41.184889+0000","last_fullsized":"2023-07-06T01:21:41.184889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:52:50.952542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.756290+0000","last_change":"2023-07-06T01:19:19.584086+0000","last_active":"2023-07-06T01:21:46.756290+0000","last_peered":"2023-07-06T01:21:46.756290+0000","last_clean":"2023-07-06T01:21:46.756290+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:21:46.756290+0000","last_undegraded":"2023-07-06T01:21:46.756290+0000","last_fullsized":"2023-07-06T01:21:46.756290+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:03.031857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":50,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.725480+0000","last_change":"2023-07-06T01:19:19.586424+0000","last_active":"2023-07-06T01:21:46.725480+0000","last_peered":"2023-07-06T01:21:46.725480+0000","last_clean":"2023-07-06T01:21:46.725480+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:21:46.725480+0000","last_undegraded":"2023-07-06T01:21:46.725480+0000","last_fullsized":"2023-07-06T01:21:46.725480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:08:16.360359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":50,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.725266+0000","last_change":"2023-07-06T01:19:19.583699+0000","last_active":"2023-07-06T01:21:46.725266+0000","last_peered":"2023-07-06T01:21:46.725266+0000","last_clean":"2023-07-06T01:21:46.725266+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:21:46.725266+0000","last_undegraded":"2023-07-06T01:21:46.725266+0000","last_fullsized":"2023-07-06T01:21:46.725266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:16:36.301039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542177,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50108,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321732,"statfs":{"total":96636764160,"available":96585453568,"internally_reserved":0,"allocated":1036288,"data_stored":774949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46500000000000002,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.623,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.497},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48999999999999999,"15min":0.49199999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69699999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.55500000000000005,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.61499999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.42699999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72199999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.439},{"interface":"front","average":{"1min":0.505,"5min":0.58299999999999996,"15min":0.59599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69499999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.746},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.68300000000000005,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.66100000000000003,"15min":0.67700000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71899999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67700000000000005}]}]},{"osd":4,"up_from":14,"seq":60129542177,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60880,"kb_used_data":7688,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94310960,"statfs":{"total":96636764160,"available":96574423040,"internally_reserved":0,"allocated":7872512,"data_stored":7612965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68600000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.38900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68799999999999994,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.443}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.502},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.74299999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79400000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57499999999999996}]},{"osd":5,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.32800000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72199999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.68500000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79400000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70799999999999996}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64600000000000002,"15min":0.65800000000000003},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81200000000000006,"5min":0.872,"15min":0.872},"last":0.68600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36188,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335652,"statfs":{"total":96636764160,"available":96599707648,"internally_reserved":0,"allocated":1069056,"data_stored":792978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":1.2589999999999999,"15min":1.4139999999999999},"min":{"1min":0.23200000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.51300000000000001,"5min":12.23,"15min":12.23},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.312,"5min":1.5329999999999999,"15min":1.7370000000000001},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.375,"5min":15.208,"15min":15.208},"last":0.31900000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.89900000000000002,"15min":0.98499999999999999},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":1.5189999999999999,"5min":12.074,"15min":12.074},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":1.306,"15min":1.458},"min":{"1min":0.26500000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73999999999999999,"5min":11.988,"15min":11.988},"last":0.34799999999999998}]},{"osd":3,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":1.8120000000000001,"15min":2.052},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.47999999999999998,"5min":18.963000000000001,"15min":18.963000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":1.5800000000000001,"15min":1.7849999999999999},"min":{"1min":0.24399999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.41299999999999998,"5min":15.244999999999999,"15min":15.244999999999999},"last":0.373}]},{"osd":4,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":1.127,"15min":1.2010000000000001},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.6080000000000001,"5min":11.923,"15min":11.923},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":1.5249999999999999,"15min":1.651},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.5950000000000002,"5min":12.019,"15min":12.019},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":1.099,"15min":1.1850000000000001},"min":{"1min":0.40100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79700000000000004,"5min":11.999000000000001,"15min":11.999000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":1.5,"15min":1.6499999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88800000000000001,"5min":11.951000000000001,"15min":11.951000000000001},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":1.788,"15min":1.9690000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.7490000000000001,"5min":15.167999999999999,"15min":15.167999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":1.1479999999999999,"15min":1.236},"min":{"1min":0.47799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":12.041,"15min":12.041},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":1.8180000000000001,"15min":2.008},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.625,"5min":15.115,"15min":15.115},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":1.163,"15min":1.2410000000000001},"min":{"1min":0.45700000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8200000000000001,"5min":11.965,"15min":11.965},"last":0.63100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.35499999999999998,"15min":0.35999999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60199999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45400000000000001,"15min":0.45500000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66800000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0880000000000001,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.56200000000000006}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.45400000000000001,"15min":0.46400000000000002},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.88800000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.496},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.45700000000000002,"15min":0.46500000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91400000000000003,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.41499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.97199999999999998,"5min":2.097,"15min":2.097},"last":0.629},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.62},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.196,"5min":0.71699999999999997,"15min":0.63700000000000001},"min":{"1min":0.44,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":11.619,"5min":11.619,"15min":11.619},"last":0.59399999999999997}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.80600000000000005,"15min":0.73199999999999998},"min":{"1min":0.50800000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":11.603999999999999,"5min":11.603999999999999,"15min":11.603999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.93899999999999995,"15min":0.98399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.518,"5min":6.3049999999999997,"15min":6.3049999999999997},"last":0.86599999999999999}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.94799999999999995,"15min":0.99099999999999999},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.609,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.2430000000000001,"5min":0.74399999999999999,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":11.663,"5min":11.663,"15min":11.663},"last":0.66000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35592,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336248,"statfs":{"total":96636764160,"available":96600317952,"internally_reserved":0,"allocated":458752,"data_stored":196675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.25700000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.497},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.33900000000000002},"min":{"1min":0.28799999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.371,"15min":0.36699999999999999},"min":{"1min":0.27100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.34200000000000003,"15min":0.33200000000000002},"min":{"1min":0.28899999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.42399999999999999,"15min":0.42199999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61199999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.47899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.47099999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.625},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35592,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336248,"statfs":{"total":96636764160,"available":96600317952,"internally_reserved":0,"allocated":458752,"data_stored":196675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.45300000000000001,"15min":0.46999999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67800000000000005,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.46200000000000002,"15min":0.47799999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.52800000000000002,"5min":1.51,"15min":1.51},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.25700000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64800000000000002,"5min":1.331,"15min":1.331},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.53400000000000003,"15min":0.56000000000000005},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.58099999999999996,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.53700000000000003,"15min":0.56100000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53600000000000003,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.376},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.53700000000000003,"15min":0.56499999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.501,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.67500000000000004,"15min":0.69499999999999995},"min":{"1min":0.44700000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.71999999999999997,"5min":1.663,"15min":1.663},"last":0.621},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.66500000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71499999999999997,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.64200000000000002,"15min":0.65800000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70199999999999996,"5min":1.667,"15min":1.667},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.68400000000000005,"15min":0.69999999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77500000000000002,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.65000000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.65700000000000003,"15min":0.67400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78100000000000003,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.78800000000000003,"15min":0.82599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76500000000000001,"5min":3.8130000000000002,"15min":3.8130000000000002},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.72399999999999998,"15min":0.753},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72599999999999998,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.75900000000000001,"15min":0.78200000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92600000000000005,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.51200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34248,"kb_used_data":3200,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94337592,"statfs":{"total":96636764160,"available":96601694208,"internally_reserved":0,"allocated":3276800,"data_stored":3014907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.499},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51700000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61899999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.36199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63100000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.432},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.53300000000000003,"15min":0.54400000000000004},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.61699999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73099999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.59399999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81200000000000006,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.58299999999999996,"15min":0.59499999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.627}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.67600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35580,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336260,"statfs":{"total":96636764160,"available":96600330240,"internally_reserved":0,"allocated":446464,"data_stored":190843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.497},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.503,"15min":0.503},"min":{"1min":0.38200000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66700000000000004,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.33000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.64600000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.56200000000000006}]},{"osd":2,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67400000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.54100000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68600000000000005,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.66200000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54100000000000004,"15min":0.54600000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67800000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.63200000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.63300000000000001}]},{"osd":7,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71199999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.745},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.66000000000000003,"5min":0.87,"15min":0.87},"last":0.67900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:21:49.792 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-06T01:21:49.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:21:49.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.915+0000 7fe862658700 1 Processor -- start 2023-07-06T01:21:49.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.916+0000 7fe862658700 1 -- start start 2023-07-06T01:21:49.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.916+0000 7fe862658700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe85c15c940 0x7fe85c0d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:49.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.916+0000 7fe862658700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe85c0cd510 0x7fe85c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:49.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.916+0000 7fe862658700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85c15cd10 con 0x7fe85c0d0fd0 2023-07-06T01:21:49.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.916+0000 7fe862658700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85c09c8a0 con 0x7fe85c0cd510 2023-07-06T01:21:49.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.916+0000 7fe862658700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0d1450 con 0x7fe85c15c940 2023-07-06T01:21:49.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.916+0000 7fe85b7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe85c0cd510 0x7fe85c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:49.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.916+0000 7fe85b7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe85c0cd510 0x7fe85c0d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48160/0 (socket says 172.21.15.22:48160) 2023-07-06T01:21:49.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.917+0000 7fe85b7fe700 1 -- 172.21.15.22:0/1713448606 learned_addr learned my addr 172.21.15.22:0/1713448606 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:49.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.917+0000 7fe85bfff700 1 --2- 172.21.15.22:0/1713448606 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe85c15c940 0x7fe85c0d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:49.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.917+0000 7fe85b7fe700 1 -- 172.21.15.22:0/1713448606 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe85c15c940 msgr2=0x7fe85c0d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.917+0000 7fe85b7fe700 1 --2- 172.21.15.22:0/1713448606 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe85c15c940 0x7fe85c0d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.917+0000 7fe85b7fe700 1 -- 172.21.15.22:0/1713448606 >> v1:172.21.15.22:6789/0 conn(0x7fe85c0d0fd0 legacy=0x7fe85c1603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.917+0000 7fe85affd700 1 -- 172.21.15.22:0/1713448606 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842328420 0 0) 0x7fe85c15cd10 con 0x7fe85c0d0fd0 2023-07-06T01:21:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.917+0000 7fe85b7fe700 1 -- 172.21.15.22:0/1713448606 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c0ce710 con 0x7fe85c0cd510 2023-07-06T01:21:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.917+0000 7fe85b7fe700 1 --2- 172.21.15.22:0/1713448606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c0cd510 0x7fe85c0d0a90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0098f0 tx=0x7fe84c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da4fa5f7dd0a4c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:49.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.917+0000 7fe85affd700 1 -- 172.21.15.22:0/1713448606 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00b8a0 con 0x7fe85c0cd510 2023-07-06T01:21:49.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.917+0000 7fe85affd700 1 -- 172.21.15.22:0/1713448606 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c00ba00 con 0x7fe85c0cd510 2023-07-06T01:21:49.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.917+0000 7fe85affd700 1 -- 172.21.15.22:0/1713448606 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00bd90 con 0x7fe85c0cd510 2023-07-06T01:21:49.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.918+0000 7fe862658700 1 -- 172.21.15.22:0/1713448606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c0cd510 msgr2=0x7fe85c0d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:49.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.918+0000 7fe862658700 1 --2- 172.21.15.22:0/1713448606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c0cd510 0x7fe85c0d0a90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0098f0 tx=0x7fe84c009c00 comp rx=0 tx=0).stop 2023-07-06T01:21:49.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.918+0000 7fe862658700 1 -- 172.21.15.22:0/1713448606 shutdown_connections 2023-07-06T01:21:49.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.918+0000 7fe862658700 1 --2- 172.21.15.22:0/1713448606 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe85c15c940 0x7fe85c0d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.918+0000 7fe862658700 1 --2- 172.21.15.22:0/1713448606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c0cd510 0x7fe85c0d0a90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.918+0000 7fe862658700 1 -- 172.21.15.22:0/1713448606 >> 172.21.15.22:0/1713448606 conn(0x7fe85c09c2e0 msgr2=0x7fe85c0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:49.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.918+0000 7fe862658700 1 -- 172.21.15.22:0/1713448606 shutdown_connections 2023-07-06T01:21:49.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.918+0000 7fe862658700 1 -- 172.21.15.22:0/1713448606 wait complete. 2023-07-06T01:21:49.926 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.919+0000 7fe862658700 1 Processor -- start 2023-07-06T01:21:49.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.919+0000 7fe862658700 1 -- start start 2023-07-06T01:21:49.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe862658700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe85c0cd510 0x7fe85c15de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:49.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe862658700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe85c0d0fd0 0x7fe85c15e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:49.927 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe862658700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c15c940 0x7fe85c157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe862658700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c15eba0 con 0x7fe85c15c940 2023-07-06T01:21:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe862658700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c15f9c0 con 0x7fe85c0cd510 2023-07-06T01:21:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe862658700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c15edb0 con 0x7fe85c0d0fd0 2023-07-06T01:21:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe860bf5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c15c940 0x7fe85c157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:49.928 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe860bf5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c15c940 0x7fe85c157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48170/0 (socket says 172.21.15.22:48170) 2023-07-06T01:21:49.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe860bf5700 1 -- 172.21.15.22:0/956075447 learned_addr learned my addr 172.21.15.22:0/956075447 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:49.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe85b7fe700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe85c0d0fd0 0x7fe85c15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:49.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe85bfff700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe85c0cd510 0x7fe85c15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:49.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe860bf5700 1 -- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe85c0d0fd0 msgr2=0x7fe85c15e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:49.929 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe860bf5700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe85c0d0fd0 0x7fe85c15e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe860bf5700 1 -- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe85c0cd510 msgr2=0x7fe85c15de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:49.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe860bf5700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe85c0cd510 0x7fe85c15de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:49.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe860bf5700 1 -- 172.21.15.22:0/956075447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c009550 con 0x7fe85c15c940 2023-07-06T01:21:49.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.920+0000 7fe85bfff700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe85c0cd510 0x7fe85c15de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:49.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.921+0000 7fe860bf5700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c15c940 0x7fe85c157380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe854001560 tx=0x7fe85400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:49.930 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.921+0000 7fe858ff9700 1 -- 172.21.15.22:0/956075447 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854003e30 con 0x7fe85c15c940 2023-07-06T01:21:49.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.921+0000 7fe858ff9700 1 -- 172.21.15.22:0/956075447 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe854005070 con 0x7fe85c15c940 2023-07-06T01:21:49.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.921+0000 7fe862658700 1 -- 172.21.15.22:0/956075447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85c15f090 con 0x7fe85c15c940 2023-07-06T01:21:49.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.921+0000 7fe858ff9700 1 -- 172.21.15.22:0/956075447 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854005420 con 0x7fe85c15c940 2023-07-06T01:21:49.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.921+0000 7fe862658700 1 -- 172.21.15.22:0/956075447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85c15f650 con 0x7fe85c15c940 2023-07-06T01:21:49.931 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.922+0000 7fe862658700 1 -- 172.21.15.22:0/956075447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85c0744b0 con 0x7fe85c15c940 2023-07-06T01:21:49.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.922+0000 7fe858ff9700 1 -- 172.21.15.22:0/956075447 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe854005580 con 0x7fe85c15c940 2023-07-06T01:21:49.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.923+0000 7fe858ff9700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe84806ad10 0x7fe84806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:49.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.923+0000 7fe858ff9700 1 -- 172.21.15.22:0/956075447 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe8540889b0 con 0x7fe85c15c940 2023-07-06T01:21:49.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.923+0000 7fe85bfff700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe84806ad10 0x7fe84806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:49.932 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.923+0000 7fe85bfff700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe84806ad10 0x7fe84806d1c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe850009a00 tx=0x7fe850008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:49.933 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:49.927+0000 7fe858ff9700 1 -- 172.21.15.22:0/956075447 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe854053cd0 con 0x7fe85c15c940 2023-07-06T01:21:50.124 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.122+0000 7fe862658700 1 -- 172.21.15.22:0/956075447 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe85c0cecd0 con 0x7fe84806ad10 2023-07-06T01:21:50.127 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.125+0000 7fe858ff9700 1 -- 172.21.15.22:0/956075447 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73494 (secure 0 0 0) 0x7fe850009610 con 0x7fe84806ad10 2023-07-06T01:21:50.128 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:21:50.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.127+0000 7fe862658700 1 -- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe84806ad10 msgr2=0x7fe84806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:50.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.127+0000 7fe862658700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe84806ad10 0x7fe84806d1c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe850009a00 tx=0x7fe850008040 comp rx=0 tx=0).stop 2023-07-06T01:21:50.129 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.127+0000 7fe862658700 1 -- 172.21.15.22:0/956075447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c15c940 msgr2=0x7fe85c157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:50.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.127+0000 7fe862658700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c15c940 0x7fe85c157380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe854001560 tx=0x7fe85400cc00 comp rx=0 tx=0).stop 2023-07-06T01:21:50.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.127+0000 7fe862658700 1 -- 172.21.15.22:0/956075447 shutdown_connections 2023-07-06T01:21:50.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.127+0000 7fe862658700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe85c15c940 0x7fe85c157380 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:50.130 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.127+0000 7fe862658700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fe84806ad10 0x7fe84806d1c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:50.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.127+0000 7fe862658700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe85c0cd510 0x7fe85c15de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:50.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.127+0000 7fe862658700 1 --2- 172.21.15.22:0/956075447 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe85c0d0fd0 0x7fe85c15e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:50.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.127+0000 7fe862658700 1 -- 172.21.15.22:0/956075447 >> 172.21.15.22:0/956075447 conn(0x7fe85c09c2e0 msgr2=0x7fe85c0aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:50.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.127+0000 7fe862658700 1 -- 172.21.15.22:0/956075447 shutdown_connections 2023-07-06T01:21:50.131 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.127+0000 7fe862658700 1 -- 172.21.15.22:0/956075447 wait complete. 2023-07-06T01:21:50.132 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:21:50.141 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2023-07-06T01:21:50.035648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":316604,"kb_used_data":7548,"kb_used_omap":0,"kb_used_meta":308992,"kb_avail":754658116,"statfs":{"total":773094113280,"available":772769910784,"internally_reserved":0,"allocated":7729152,"data_stored":5590276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316407808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7430132,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-2114,"num_read_kb":-2464,"num_write":-17718,"num_write_kb":-101964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-29,"num_bytes_recovered":-94,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.334214"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.752615+0000","last_change":"2023-07-06T01:19:20.074195+0000","last_active":"2023-07-06T01:21:46.752615+0000","last_peered":"2023-07-06T01:21:46.752615+0000","last_clean":"2023-07-06T01:21:46.752615+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:21:46.752615+0000","last_undegraded":"2023-07-06T01:21:46.752615+0000","last_fullsized":"2023-07-06T01:21:46.752615+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:58:56.446322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.754269+0000","last_change":"2023-07-06T01:19:19.587110+0000","last_active":"2023-07-06T01:21:46.754269+0000","last_peered":"2023-07-06T01:21:46.754269+0000","last_clean":"2023-07-06T01:21:46.754269+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:21:46.754269+0000","last_undegraded":"2023-07-06T01:21:46.754269+0000","last_fullsized":"2023-07-06T01:21:46.754269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:38.656516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":50,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.760757+0000","last_change":"2023-07-06T01:19:19.584242+0000","last_active":"2023-07-06T01:21:46.760757+0000","last_peered":"2023-07-06T01:21:46.760757+0000","last_clean":"2023-07-06T01:21:46.760757+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:21:46.760757+0000","last_undegraded":"2023-07-06T01:21:46.760757+0000","last_fullsized":"2023-07-06T01:21:46.760757+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:26:02.484123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":202,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.724661+0000","last_change":"2023-07-06T01:19:19.586495+0000","last_active":"2023-07-06T01:21:46.724661+0000","last_peered":"2023-07-06T01:21:46.724661+0000","last_clean":"2023-07-06T01:21:46.724661+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:21:46.724661+0000","last_undegraded":"2023-07-06T01:21:46.724661+0000","last_fullsized":"2023-07-06T01:21:46.724661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:38.656516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":52,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.721628+0000","last_change":"2023-07-06T01:19:19.586426+0000","last_active":"2023-07-06T01:21:46.721628+0000","last_peered":"2023-07-06T01:21:46.721628+0000","last_clean":"2023-07-06T01:21:46.721628+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:21:46.721628+0000","last_undegraded":"2023-07-06T01:21:46.721628+0000","last_fullsized":"2023-07-06T01:21:46.721628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:40:48.935924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":52,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.724410+0000","last_change":"2023-07-06T01:19:19.586626+0000","last_active":"2023-07-06T01:21:46.724410+0000","last_peered":"2023-07-06T01:21:46.724410+0000","last_clean":"2023-07-06T01:21:46.724410+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:21:46.724410+0000","last_undegraded":"2023-07-06T01:21:46.724410+0000","last_fullsized":"2023-07-06T01:21:46.724410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:27.309882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.756290+0000","last_change":"2023-07-06T01:19:19.584086+0000","last_active":"2023-07-06T01:21:46.756290+0000","last_peered":"2023-07-06T01:21:46.756290+0000","last_clean":"2023-07-06T01:21:46.756290+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:21:46.756290+0000","last_undegraded":"2023-07-06T01:21:46.756290+0000","last_fullsized":"2023-07-06T01:21:46.756290+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:03.031857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":50,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.725480+0000","last_change":"2023-07-06T01:19:19.586424+0000","last_active":"2023-07-06T01:21:46.725480+0000","last_peered":"2023-07-06T01:21:46.725480+0000","last_clean":"2023-07-06T01:21:46.725480+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:21:46.725480+0000","last_undegraded":"2023-07-06T01:21:46.725480+0000","last_fullsized":"2023-07-06T01:21:46.725480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:08:16.360359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":50,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.725266+0000","last_change":"2023-07-06T01:19:19.583699+0000","last_active":"2023-07-06T01:21:46.725266+0000","last_peered":"2023-07-06T01:21:46.725266+0000","last_clean":"2023-07-06T01:21:46.725266+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:21:46.725266+0000","last_undegraded":"2023-07-06T01:21:46.725266+0000","last_fullsized":"2023-07-06T01:21:46.725266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:16:36.301039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50140,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321700,"statfs":{"total":96636764160,"available":96585420800,"internally_reserved":0,"allocated":1069056,"data_stored":792978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46500000000000002,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.623,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.66400000000000003}]},{"osd":1,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.497},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.69299999999999995}]},{"osd":2,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48999999999999999,"15min":0.49199999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69699999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.55500000000000005,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.61499999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.42699999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72199999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.433},{"interface":"front","average":{"1min":0.505,"5min":0.58299999999999996,"15min":0.59599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69499999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.68300000000000005,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.66100000000000003,"15min":0.67700000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71899999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.33600000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53656,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94318184,"statfs":{"total":96636764160,"available":96581820416,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68600000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.47199999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.38900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68799999999999994,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.74299999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79400000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.32800000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72199999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.68500000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79400000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64600000000000002,"15min":0.65800000000000003},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81200000000000006,"5min":0.872,"15min":0.872},"last":0.65200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36188,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335652,"statfs":{"total":96636764160,"available":96599707648,"internally_reserved":0,"allocated":1069056,"data_stored":792978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":1.2589999999999999,"15min":1.4139999999999999},"min":{"1min":0.23200000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.51300000000000001,"5min":12.23,"15min":12.23},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.312,"5min":1.5329999999999999,"15min":1.7370000000000001},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.375,"5min":15.208,"15min":15.208},"last":0.31900000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.89900000000000002,"15min":0.98499999999999999},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":1.5189999999999999,"5min":12.074,"15min":12.074},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":1.306,"15min":1.458},"min":{"1min":0.26500000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73999999999999999,"5min":11.988,"15min":11.988},"last":0.34799999999999998}]},{"osd":3,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":1.8120000000000001,"15min":2.052},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.47999999999999998,"5min":18.963000000000001,"15min":18.963000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":1.5800000000000001,"15min":1.7849999999999999},"min":{"1min":0.24399999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.41299999999999998,"5min":15.244999999999999,"15min":15.244999999999999},"last":0.373}]},{"osd":4,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":1.127,"15min":1.2010000000000001},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.6080000000000001,"5min":11.923,"15min":11.923},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":1.5249999999999999,"15min":1.651},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.5950000000000002,"5min":12.019,"15min":12.019},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":1.099,"15min":1.1850000000000001},"min":{"1min":0.40100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79700000000000004,"5min":11.999000000000001,"15min":11.999000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":1.5,"15min":1.6499999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88800000000000001,"5min":11.951000000000001,"15min":11.951000000000001},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":1.788,"15min":1.9690000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.7490000000000001,"5min":15.167999999999999,"15min":15.167999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":1.1479999999999999,"15min":1.236},"min":{"1min":0.47799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":12.041,"15min":12.041},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":1.8180000000000001,"15min":2.008},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.625,"5min":15.115,"15min":15.115},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":1.163,"15min":1.2410000000000001},"min":{"1min":0.45700000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8200000000000001,"5min":11.965,"15min":11.965},"last":0.63100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.35499999999999998,"15min":0.35999999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60199999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45400000000000001,"15min":0.45500000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66800000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0880000000000001,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.56200000000000006}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.45400000000000001,"15min":0.46400000000000002},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.88800000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.496},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.45700000000000002,"15min":0.46500000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91400000000000003,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.41499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.97199999999999998,"5min":2.097,"15min":2.097},"last":0.629},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.62},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.196,"5min":0.71699999999999997,"15min":0.63700000000000001},"min":{"1min":0.44,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":11.619,"5min":11.619,"15min":11.619},"last":0.59399999999999997}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.80600000000000005,"15min":0.73199999999999998},"min":{"1min":0.50800000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":11.603999999999999,"5min":11.603999999999999,"15min":11.603999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.93899999999999995,"15min":0.98399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.518,"5min":6.3049999999999997,"15min":6.3049999999999997},"last":0.86599999999999999}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.94799999999999995,"15min":0.99099999999999999},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.609,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.2430000000000001,"5min":0.74399999999999999,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":11.663,"5min":11.663,"15min":11.663},"last":0.66000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35592,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336248,"statfs":{"total":96636764160,"available":96600317952,"internally_reserved":0,"allocated":458752,"data_stored":196675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.25700000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.497},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.33900000000000002},"min":{"1min":0.28799999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.371,"15min":0.36699999999999999},"min":{"1min":0.27100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.34200000000000003,"15min":0.33200000000000002},"min":{"1min":0.28899999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.42399999999999999,"15min":0.42199999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61199999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.47899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.47099999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.625},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35592,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336248,"statfs":{"total":96636764160,"available":96600317952,"internally_reserved":0,"allocated":458752,"data_stored":196675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.45300000000000001,"15min":0.46999999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67800000000000005,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.46200000000000002,"15min":0.47799999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.52800000000000002,"5min":1.51,"15min":1.51},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.25700000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64800000000000002,"5min":1.331,"15min":1.331},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.53400000000000003,"15min":0.56000000000000005},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.58099999999999996,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.53700000000000003,"15min":0.56100000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53600000000000003,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.376},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.53700000000000003,"15min":0.56499999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.501,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.67500000000000004,"15min":0.69499999999999995},"min":{"1min":0.44700000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.71999999999999997,"5min":1.663,"15min":1.663},"last":0.621},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.66500000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71499999999999997,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.64200000000000002,"15min":0.65800000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70199999999999996,"5min":1.667,"15min":1.667},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.68400000000000005,"15min":0.69999999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77500000000000002,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.65000000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.65700000000000003,"15min":0.67400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78100000000000003,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.78800000000000003,"15min":0.82599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76500000000000001,"5min":3.8130000000000002,"15min":3.8130000000000002},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.72399999999999998,"15min":0.753},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72599999999999998,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.75900000000000001,"15min":0.78200000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92600000000000005,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.51200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34248,"kb_used_data":3200,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94337592,"statfs":{"total":96636764160,"available":96601694208,"internally_reserved":0,"allocated":3276800,"data_stored":3014907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.499},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51700000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61899999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.36199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63100000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.432},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.53300000000000003,"15min":0.54400000000000004},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.61699999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73099999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.59399999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81200000000000006,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.58299999999999996,"15min":0.59499999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.627}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.67600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35580,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336260,"statfs":{"total":96636764160,"available":96600330240,"internally_reserved":0,"allocated":446464,"data_stored":190843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.497},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.503,"15min":0.503},"min":{"1min":0.38200000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66700000000000004,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.33000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.64600000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.56200000000000006}]},{"osd":2,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67400000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.54100000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68600000000000005,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.66200000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54100000000000004,"15min":0.54600000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67800000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.63200000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.63300000000000001}]},{"osd":7,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71199999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.745},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.66000000000000003,"5min":0.87,"15min":0.87},"last":0.67900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:21:50.146 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T01:21:50.270 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.269+0000 7f5a6c647700 1 Processor -- start 2023-07-06T01:21:50.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.269+0000 7f5a6c647700 1 -- start start 2023-07-06T01:21:50.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a6c647700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5a6415a710 0x7f5a640ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:50.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a6c647700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a640aecb0 0x7f5a640ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:50.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a6c647700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a640af270 con 0x7f5a640ceda0 2023-07-06T01:21:50.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a6c647700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a640cf220 con 0x7f5a6415a710 2023-07-06T01:21:50.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a6c647700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640af550 con 0x7f5a640aecb0 2023-07-06T01:21:50.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a6a3e3700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5a6415a710 0x7f5a640ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:50.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a6a3e3700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5a6415a710 0x7f5a640ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48174/0 (socket says 172.21.15.22:48174) 2023-07-06T01:21:50.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a6a3e3700 1 -- 172.21.15.22:0/1881372435 learned_addr learned my addr 172.21.15.22:0/1881372435 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:50.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a69be2700 1 --2- 172.21.15.22:0/1881372435 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a640aecb0 0x7f5a640ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:50.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a6a3e3700 1 -- 172.21.15.22:0/1881372435 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a640aecb0 msgr2=0x7f5a640ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:50.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a6a3e3700 1 --2- 172.21.15.22:0/1881372435 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a640aecb0 0x7f5a640ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:50.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a6a3e3700 1 -- 172.21.15.22:0/1881372435 >> v1:172.21.15.103:6789/0 conn(0x7f5a640ceda0 legacy=0x7f5a6415e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:50.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a6a3e3700 1 -- 172.21.15.22:0/1881372435 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a640af830 con 0x7f5a6415a710 2023-07-06T01:21:50.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a6a3e3700 1 --2- 172.21.15.22:0/1881372435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a6415a710 0x7f5a640ae770 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5a600098f0 tx=0x7f5a60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a30b883e098845 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:50.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a693e1700 1 -- 172.21.15.22:0/1881372435 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000b8a0 con 0x7f5a6415a710 2023-07-06T01:21:50.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a693e1700 1 -- 172.21.15.22:0/1881372435 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6000ba00 con 0x7f5a6415a710 2023-07-06T01:21:50.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.270+0000 7f5a693e1700 1 -- 172.21.15.22:0/1881372435 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000bd90 con 0x7f5a6415a710 2023-07-06T01:21:50.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.271+0000 7f5a6c647700 1 -- 172.21.15.22:0/1881372435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a6415a710 msgr2=0x7f5a640ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:50.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.271+0000 7f5a6c647700 1 --2- 172.21.15.22:0/1881372435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a6415a710 0x7f5a640ae770 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5a600098f0 tx=0x7f5a60009c00 comp rx=0 tx=0).stop 2023-07-06T01:21:50.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.271+0000 7f5a6c647700 1 -- 172.21.15.22:0/1881372435 shutdown_connections 2023-07-06T01:21:50.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.271+0000 7f5a6c647700 1 --2- 172.21.15.22:0/1881372435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a6415a710 0x7f5a640ae770 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:50.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.271+0000 7f5a6c647700 1 --2- 172.21.15.22:0/1881372435 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a640aecb0 0x7f5a640ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:50.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.271+0000 7f5a6c647700 1 -- 172.21.15.22:0/1881372435 >> 172.21.15.22:0/1881372435 conn(0x7f5a640999f0 msgr2=0x7f5a64099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:50.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.271+0000 7f5a6c647700 1 -- 172.21.15.22:0/1881372435 shutdown_connections 2023-07-06T01:21:50.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.271+0000 7f5a6c647700 1 -- 172.21.15.22:0/1881372435 wait complete. 2023-07-06T01:21:50.280 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.272+0000 7f5a6c647700 1 Processor -- start 2023-07-06T01:21:50.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.272+0000 7f5a6c647700 1 -- start start 2023-07-06T01:21:50.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.273+0000 7f5a6c647700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a640aecb0 0x7f5a641460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:50.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.273+0000 7f5a6c647700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a640ceda0 0x7f5a640c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:50.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.273+0000 7f5a6c647700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a6415a710 0x7f5a640b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:50.281 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.273+0000 7f5a6c647700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640c4a60 con 0x7f5a640aecb0 2023-07-06T01:21:50.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.273+0000 7f5a6c647700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640ca710 con 0x7f5a6415a710 2023-07-06T01:21:50.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.273+0000 7f5a6c647700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640ca9f0 con 0x7f5a640ceda0 2023-07-06T01:21:50.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.273+0000 7f5a6a3e3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a640aecb0 0x7f5a641460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:50.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.273+0000 7f5a6a3e3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a640aecb0 0x7f5a641460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48178/0 (socket says 172.21.15.22:48178) 2023-07-06T01:21:50.282 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.273+0000 7f5a6a3e3700 1 -- 172.21.15.22:0/727040681 learned_addr learned my addr 172.21.15.22:0/727040681 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:50.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.273+0000 7f5a69be2700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a640ceda0 0x7f5a640c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:50.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.273+0000 7f5a6abe4700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a6415a710 0x7f5a640b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:50.283 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.274+0000 7f5a6a3e3700 1 -- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a640ceda0 msgr2=0x7f5a640c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:50.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.274+0000 7f5a6a3e3700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a640ceda0 0x7f5a640c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:50.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.274+0000 7f5a6a3e3700 1 -- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a6415a710 msgr2=0x7f5a640b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:50.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.274+0000 7f5a6a3e3700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a6415a710 0x7f5a640b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:50.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.274+0000 7f5a6a3e3700 1 -- 172.21.15.22:0/727040681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a60009550 con 0x7f5a640aecb0 2023-07-06T01:21:50.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.274+0000 7f5a6a3e3700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a640aecb0 0x7f5a641460a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5a60000c00 tx=0x7f5a60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:50.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.274+0000 7f5a5affd700 1 -- 172.21.15.22:0/727040681 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60004400 con 0x7f5a640aecb0 2023-07-06T01:21:50.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.274+0000 7f5a5affd700 1 -- 172.21.15.22:0/727040681 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6001f070 con 0x7f5a640aecb0 2023-07-06T01:21:50.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.274+0000 7f5a6c647700 1 -- 172.21.15.22:0/727040681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a640cacd0 con 0x7f5a640aecb0 2023-07-06T01:21:50.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.274+0000 7f5a5affd700 1 -- 172.21.15.22:0/727040681 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a600038e0 con 0x7f5a640aecb0 2023-07-06T01:21:50.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.274+0000 7f5a6c647700 1 -- 172.21.15.22:0/727040681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a640ca3d0 con 0x7f5a640aecb0 2023-07-06T01:21:50.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.275+0000 7f5a6c647700 1 -- 172.21.15.22:0/727040681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a640c47c0 con 0x7f5a640aecb0 2023-07-06T01:21:50.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.275+0000 7f5a5affd700 1 -- 172.21.15.22:0/727040681 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a60021030 con 0x7f5a640aecb0 2023-07-06T01:21:50.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.276+0000 7f5a5affd700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:50.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.276+0000 7f5a5affd700 1 -- 172.21.15.22:0/727040681 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5a6008a5d0 con 0x7f5a640aecb0 2023-07-06T01:21:50.286 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.276+0000 7f5a69be2700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:50.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.276+0000 7f5a69be2700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5a4806ace0 0x7f5a4806d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5a641f3820 tx=0x7f5a54008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:50.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.280+0000 7f5a5affd700 1 -- 172.21.15.22:0/727040681 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5a640c47c0 con 0x7f5a640aecb0 2023-07-06T01:21:50.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.480+0000 7f5a6c647700 1 -- 172.21.15.22:0/727040681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a640c47c0 con 0x7f5a640aecb0 2023-07-06T01:21:50.482 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.481+0000 7f5a5affd700 1 -- 172.21.15.22:0/727040681 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+14696 (secure 0 0 0) 0x7f5a640c47c0 con 0x7f5a640aecb0 2023-07-06T01:21:50.483 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:21:50.483 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":41,"fsid":"63a894dd-9157-4ce0-adf3-c1bdfe16ec64","created":"2023-07-06T01:19:06.400410+0000","modified":"2023-07-06T01:21:46.707048+0000","last_up_change":"2023-07-06T01:19:17.235081+0000","last_in_change":"2023-07-06T01:19:08.827237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-07-06T01:19:17.673760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-07-06T01:19:18.054792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a2ddc168-d6c5-4312-b30d-71c083434002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6809","nonce":683380483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6811","nonce":683380483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6815","nonce":683380483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":683380483},{"type":"v1","addr":"172.21.15.22:6813","nonce":683380483}]},"public_addr":"172.21.15.22:6809/683380483","cluster_addr":"172.21.15.22:6811/683380483","heartbeat_back_addr":"172.21.15.22:6815/683380483","heartbeat_front_addr":"172.21.15.22:6813/683380483","state":["exists","up"]},{"osd":1,"uuid":"f25f28cf-f149-4a01-b035-e9104cb10927","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6801","nonce":2676174112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6803","nonce":2676174112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6807","nonce":2676174112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":2676174112},{"type":"v1","addr":"172.21.15.22:6805","nonce":2676174112}]},"public_addr":"172.21.15.22:6801/2676174112","cluster_addr":"172.21.15.22:6803/2676174112","heartbeat_back_addr":"172.21.15.22:6807/2676174112","heartbeat_front_addr":"172.21.15.22:6805/2676174112","state":["exists","up"]},{"osd":2,"uuid":"a447ebab-c06d-4d60-a53c-e6847032aeba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6817","nonce":579920713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6819","nonce":579920713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6823","nonce":579920713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":579920713},{"type":"v1","addr":"172.21.15.22:6821","nonce":579920713}]},"public_addr":"172.21.15.22:6817/579920713","cluster_addr":"172.21.15.22:6819/579920713","heartbeat_back_addr":"172.21.15.22:6823/579920713","heartbeat_front_addr":"172.21.15.22:6821/579920713","state":["exists","up"]},{"osd":3,"uuid":"046c24dd-f90c-4dab-b24b-db82361d3c52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6825","nonce":3305852500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6827","nonce":3305852500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6831","nonce":3305852500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":3305852500},{"type":"v1","addr":"172.21.15.22:6829","nonce":3305852500}]},"public_addr":"172.21.15.22:6825/3305852500","cluster_addr":"172.21.15.22:6827/3305852500","heartbeat_back_addr":"172.21.15.22:6831/3305852500","heartbeat_front_addr":"172.21.15.22:6829/3305852500","state":["exists","up"]},{"osd":4,"uuid":"dd31524d-cdf4-42da-a0d4-14962f66fbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6825","nonce":3712184942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6827","nonce":3712184942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6831","nonce":3712184942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":3712184942},{"type":"v1","addr":"172.21.15.103:6829","nonce":3712184942}]},"public_addr":"172.21.15.103:6825/3712184942","cluster_addr":"172.21.15.103:6827/3712184942","heartbeat_back_addr":"172.21.15.103:6831/3712184942","heartbeat_front_addr":"172.21.15.103:6829/3712184942","state":["exists","up"]},{"osd":5,"uuid":"7902da34-cb61-4494-a428-e23f242846bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6801","nonce":1893069563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6803","nonce":1893069563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6807","nonce":1893069563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1893069563},{"type":"v1","addr":"172.21.15.103:6805","nonce":1893069563}]},"public_addr":"172.21.15.103:6801/1893069563","cluster_addr":"172.21.15.103:6803/1893069563","heartbeat_back_addr":"172.21.15.103:6807/1893069563","heartbeat_front_addr":"172.21.15.103:6805/1893069563","state":["exists","up"]},{"osd":6,"uuid":"f9d5652b-3c91-45a7-b98a-a56a76ed9247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6809","nonce":2564178583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6811","nonce":2564178583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6815","nonce":2564178583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2564178583},{"type":"v1","addr":"172.21.15.103:6813","nonce":2564178583}]},"public_addr":"172.21.15.103:6809/2564178583","cluster_addr":"172.21.15.103:6811/2564178583","heartbeat_back_addr":"172.21.15.103:6815/2564178583","heartbeat_front_addr":"172.21.15.103:6813/2564178583","state":["exists","up"]},{"osd":7,"uuid":"bcacc64c-8ca7-4a3c-954b-99e10ce3f659","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6817","nonce":1845979485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6819","nonce":1845979485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6823","nonce":1845979485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1845979485},{"type":"v1","addr":"172.21.15.103:6821","nonce":1845979485}]},"public_addr":"172.21.15.103:6817/1845979485","cluster_addr":"172.21.15.103:6819/1845979485","heartbeat_back_addr":"172.21.15.103:6823/1845979485","heartbeat_front_addr":"172.21.15.103:6821/1845979485","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.076721+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.058018+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.040148+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.068476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.009173+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:13.953548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.023333+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T01:19:14.006955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6835/612827809":"2023-07-07T01:21:43.698998+0000","172.21.15.22:6833/612827809":"2023-07-07T01:21:43.698998+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T01:21:50.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.483+0000 7f5a6c647700 1 -- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5a4806ace0 msgr2=0x7f5a4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:50.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.483+0000 7f5a6c647700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5a4806ace0 0x7f5a4806d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5a641f3820 tx=0x7f5a54008040 comp rx=0 tx=0).stop 2023-07-06T01:21:50.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.483+0000 7f5a6c647700 1 -- 172.21.15.22:0/727040681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a640aecb0 msgr2=0x7f5a641460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:50.485 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.483+0000 7f5a6c647700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a640aecb0 0x7f5a641460a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5a60000c00 tx=0x7f5a60004750 comp rx=0 tx=0).stop 2023-07-06T01:21:50.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.483+0000 7f5a6c647700 1 -- 172.21.15.22:0/727040681 shutdown_connections 2023-07-06T01:21:50.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.483+0000 7f5a6c647700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5a4806ace0 0x7f5a4806d190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:50.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.483+0000 7f5a6c647700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a6415a710 0x7f5a640b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:50.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.483+0000 7f5a6c647700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a640aecb0 0x7f5a641460a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:50.486 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.483+0000 7f5a6c647700 1 --2- 172.21.15.22:0/727040681 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5a640ceda0 0x7f5a640c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:50.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.483+0000 7f5a6c647700 1 -- 172.21.15.22:0/727040681 >> 172.21.15.22:0/727040681 conn(0x7f5a640999f0 msgr2=0x7f5a640cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:50.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.483+0000 7f5a6c647700 1 -- 172.21.15.22:0/727040681 shutdown_connections 2023-07-06T01:21:50.487 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:50.483+0000 7f5a6c647700 1 -- 172.21.15.22:0/727040681 wait complete. 2023-07-06T01:21:51.497 INFO:tasks.ceph:Scrubbing osd.0 2023-07-06T01:21:51.498 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-07-06T01:21:51.642 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.640+0000 7f02ce60b700 1 Processor -- start 2023-07-06T01:21:51.643 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.640+0000 7f02ce60b700 1 -- start start 2023-07-06T01:21:51.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.641+0000 7f02ce60b700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f02c815cb50 0x7f02c806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.641+0000 7f02ce60b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02c806a540 0x7f02c80ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.641+0000 7f02ce60b700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c80ca2b0 con 0x7f02c80cf090 2023-07-06T01:21:51.645 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.641+0000 7f02ce60b700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c80af440 con 0x7f02c815cb50 2023-07-06T01:21:51.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.641+0000 7f02ce60b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c806a970 con 0x7f02c806a540 2023-07-06T01:21:51.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c77fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02c806a540 0x7f02c80ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c7fff700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f02c815cb50 0x7f02c806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02ccba8700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f02c80cf090 0x7f02c806cae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:40998/0 (socket says 172.21.15.22:40998) 2023-07-06T01:21:51.646 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c77fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02c806a540 0x7f02c80ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:59326/0 (socket says 172.21.15.22:59326) 2023-07-06T01:21:51.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02ccba8700 1 -- 172.21.15.22:0/1514281842 learned_addr learned my addr 172.21.15.22:0/1514281842 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:51.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c6ffd700 1 -- 172.21.15.22:0/1514281842 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915125445 0 0) 0x7f02c80ca2b0 con 0x7f02c80cf090 2023-07-06T01:21:51.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c6ffd700 1 -- 172.21.15.22:0/1514281842 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02b40035b0 con 0x7f02c80cf090 2023-07-06T01:21:51.647 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c77fe700 1 -- 172.21.15.22:0/1514281842 >> v2:172.21.15.103:3300/0 conn(0x7f02c815cb50 msgr2=0x7f02c806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c77fe700 1 --2- 172.21.15.22:0/1514281842 >> v2:172.21.15.103:3300/0 conn(0x7f02c815cb50 0x7f02c806a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c77fe700 1 -- 172.21.15.22:0/1514281842 >> v1:172.21.15.103:6789/0 conn(0x7f02c80cf090 legacy=0x7f02c806cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c77fe700 1 -- 172.21.15.22:0/1514281842 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c80cfc90 con 0x7f02c806a540 2023-07-06T01:21:51.648 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c7fff700 1 --2- 172.21.15.22:0/1514281842 >> v2:172.21.15.103:3300/0 conn(0x7f02c815cb50 0x7f02c806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c77fe700 1 --2- 172.21.15.22:0/1514281842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02c806a540 0x7f02c80ceb50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009840 tx=0x7f02b0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b813ef43e935bfaa server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c6ffd700 1 -- 172.21.15.22:0/1514281842 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0011830 con 0x7f02c806a540 2023-07-06T01:21:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c6ffd700 1 -- 172.21.15.22:0/1514281842 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b0011990 con 0x7f02c806a540 2023-07-06T01:21:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.642+0000 7f02c6ffd700 1 -- 172.21.15.22:0/1514281842 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0011d20 con 0x7f02c806a540 2023-07-06T01:21:51.649 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.643+0000 7f02ce60b700 1 -- 172.21.15.22:0/1514281842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02c806a540 msgr2=0x7f02c80ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:51.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.643+0000 7f02ce60b700 1 --2- 172.21.15.22:0/1514281842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02c806a540 0x7f02c80ceb50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009840 tx=0x7f02b0009b50 comp rx=0 tx=0).stop 2023-07-06T01:21:51.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.643+0000 7f02ce60b700 1 -- 172.21.15.22:0/1514281842 shutdown_connections 2023-07-06T01:21:51.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.643+0000 7f02ce60b700 1 --2- 172.21.15.22:0/1514281842 >> v2:172.21.15.103:3300/0 conn(0x7f02c815cb50 0x7f02c806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:51.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.643+0000 7f02ce60b700 1 --2- 172.21.15.22:0/1514281842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02c806a540 0x7f02c80ceb50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:51.650 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.643+0000 7f02ce60b700 1 -- 172.21.15.22:0/1514281842 >> 172.21.15.22:0/1514281842 conn(0x7f02c806b110 msgr2=0x7f02c806b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:51.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.643+0000 7f02ce60b700 1 -- 172.21.15.22:0/1514281842 shutdown_connections 2023-07-06T01:21:51.651 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.643+0000 7f02ce60b700 1 -- 172.21.15.22:0/1514281842 wait complete. 2023-07-06T01:21:51.653 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.644+0000 7f02ce60b700 1 Processor -- start 2023-07-06T01:21:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.644+0000 7f02ce60b700 1 -- start start 2023-07-06T01:21:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.645+0000 7f02ce60b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02c806a540 0x7f02c815b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.645+0000 7f02ce60b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02c80cf090 0x7f02c80c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.654 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.645+0000 7f02ce60b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02c815cb50 0x7f02c806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.645+0000 7f02ce60b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c80c5da0 con 0x7f02c80cf090 2023-07-06T01:21:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.645+0000 7f02ce60b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c80c4ed0 con 0x7f02c806a540 2023-07-06T01:21:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.645+0000 7f02ce60b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c81f2e10 con 0x7f02c815cb50 2023-07-06T01:21:51.655 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.645+0000 7f02c77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02c80cf090 0x7f02c80c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.645+0000 7f02c77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02c80cf090 0x7f02c80c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48194/0 (socket says 172.21.15.22:48194) 2023-07-06T01:21:51.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.645+0000 7f02c77fe700 1 -- 172.21.15.22:0/7228197 learned_addr learned my addr 172.21.15.22:0/7228197 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:51.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.646+0000 7f02c7fff700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02c806a540 0x7f02c815b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.646+0000 7f02ccba8700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02c815cb50 0x7f02c806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.656 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.646+0000 7f02c77fe700 1 -- 172.21.15.22:0/7228197 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02c815cb50 msgr2=0x7f02c806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.646+0000 7f02c77fe700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f02c815cb50 0x7f02c806b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.646+0000 7f02c77fe700 1 -- 172.21.15.22:0/7228197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02c806a540 msgr2=0x7f02c815b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.646+0000 7f02c77fe700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02c806a540 0x7f02c815b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.646+0000 7f02c77fe700 1 -- 172.21.15.22:0/7228197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b80094a0 con 0x7f02c80cf090 2023-07-06T01:21:51.657 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.646+0000 7f02c77fe700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02c80cf090 0x7f02c80c5360 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f02b00090b0 tx=0x7f02b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.646+0000 7f02c4ff9700 1 -- 172.21.15.22:0/7228197 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b00036a0 con 0x7f02c80cf090 2023-07-06T01:21:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.646+0000 7f02c4ff9700 1 -- 172.21.15.22:0/7228197 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b001f070 con 0x7f02c80cf090 2023-07-06T01:21:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.646+0000 7f02ce60b700 1 -- 172.21.15.22:0/7228197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b00094a0 con 0x7f02c80cf090 2023-07-06T01:21:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.646+0000 7f02c4ff9700 1 -- 172.21.15.22:0/7228197 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0003a00 con 0x7f02c80cf090 2023-07-06T01:21:51.658 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.646+0000 7f02ce60b700 1 -- 172.21.15.22:0/7228197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c80c4880 con 0x7f02c80cf090 2023-07-06T01:21:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.647+0000 7f02ce60b700 1 -- 172.21.15.22:0/7228197 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f02c80d0210 con 0x7f02c80cf090 2023-07-06T01:21:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.648+0000 7f02c4ff9700 1 -- 172.21.15.22:0/7228197 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f02b0003b60 con 0x7f02c80cf090 2023-07-06T01:21:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.648+0000 7f02c4ff9700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f02a806ace0 0x7f02a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.659 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.648+0000 7f02c4ff9700 1 -- 172.21.15.22:0/7228197 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f02b0021070 con 0x7f02c80cf090 2023-07-06T01:21:51.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.648+0000 7f02c4ff9700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] conn(0x7f02a8071150 0x7f02a8073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.648+0000 7f02c4ff9700 1 -- 172.21.15.22:0/7228197 --> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02a8073c30 con 0x7f02a8071150 2023-07-06T01:21:51.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.648+0000 7f02c4ff9700 1 -- 172.21.15.22:0/7228197 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b008b080 con 0x7f02c80cf090 2023-07-06T01:21:51.660 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.648+0000 7f02c7fff700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f02a806ace0 0x7f02a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.649+0000 7f02ccba8700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] conn(0x7f02a8071150 0x7f02a8073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.649+0000 7f02c7fff700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f02a806ace0 0x7f02a806d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f02c81e7a80 tx=0x7f02b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.649+0000 7f02ccba8700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] conn(0x7f02a8071150 0x7f02a8073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:51.661 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.650+0000 7f02c4ff9700 1 -- 172.21.15.22:0/7228197 <== osd.0 v2:172.21.15.22:6808/683380483 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f02a8073c30 con 0x7f02a8071150 2023-07-06T01:21:51.665 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.664+0000 7f02ce60b700 1 -- 172.21.15.22:0/7228197 --> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f02c80cff30 con 0x7f02a8071150 2023-07-06T01:21:51.666 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.664+0000 7f02c4ff9700 1 -- 172.21.15.22:0/7228197 <== osd.0 v2:172.21.15.22:6808/683380483 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f02c00070f0 con 0x7f02a8071150 2023-07-06T01:21:51.666 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:21:51.667 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T01:21:51.667 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:21:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.664+0000 7f02ce60b700 1 -- 172.21.15.22:0/7228197 >> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] conn(0x7f02a8071150 msgr2=0x7f02a8073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.664+0000 7f02ce60b700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.22:6808/683380483,v1:172.21.15.22:6809/683380483] conn(0x7f02a8071150 0x7f02a8073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.665+0000 7f02ce60b700 1 -- 172.21.15.22:0/7228197 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f02a806ace0 msgr2=0x7f02a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:51.668 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.665+0000 7f02ce60b700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f02a806ace0 0x7f02a806d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f02c81e7a80 tx=0x7f02b8008040 comp rx=0 tx=0).stop 2023-07-06T01:21:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.665+0000 7f02ce60b700 1 -- 172.21.15.22:0/7228197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02c80cf090 msgr2=0x7f02c80c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.665+0000 7f02ce60b700 1 --2- 172.21.15.22:0/7228197 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02c80cf090 0x7f02c80c5360 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f02b00090b0 tx=0x7f02b0004750 comp rx=0 tx=0).stop 2023-07-06T01:21:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.665+0000 7f02ccba8700 1 -- 172.21.15.22:0/7228197 reap_dead start 2023-07-06T01:21:51.669 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.665+0000 7f02ce60b700 1 -- 172.21.15.22:0/7228197 shutdown_connections 2023-07-06T01:21:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.665+0000 7f02ce60b700 1 -- 172.21.15.22:0/7228197 >> 172.21.15.22:0/7228197 conn(0x7f02c806b110 msgr2=0x7f02c80b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.665+0000 7f02ce60b700 1 -- 172.21.15.22:0/7228197 shutdown_connections 2023-07-06T01:21:51.670 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.665+0000 7f02ce60b700 1 -- 172.21.15.22:0/7228197 wait complete. 2023-07-06T01:21:51.678 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-07-06T01:21:51.804 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.803+0000 7f0e73046700 1 Processor -- start 2023-07-06T01:21:51.805 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.803+0000 7f0e73046700 1 -- start start 2023-07-06T01:21:51.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.803+0000 7f0e73046700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 0x7f0e6c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.803+0000 7f0e73046700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6c0cebd0 con 0x7f0e6c0b36f0 2023-07-06T01:21:51.808 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e73046700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6c0cdfc0 con 0x7f0e6c0b3b70 2023-07-06T01:21:51.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e73046700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c15cff0 con 0x7f0e6c0d11a0 2023-07-06T01:21:51.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e70de2700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f0e6c0b36f0 0x7f0e6c0cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38946/0 (socket says 172.21.15.22:38946) 2023-07-06T01:21:51.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e70de2700 1 -- 172.21.15.22:0/1446198090 learned_addr learned my addr 172.21.15.22:0/1446198090 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:51.809 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e715e3700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 0x7f0e6c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475078682 0 0) 0x7f0e6c0cebd0 con 0x7f0e6c0b36f0 2023-07-06T01:21:51.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e5c0035b0 con 0x7f0e6c0b36f0 2023-07-06T01:21:51.810 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1806317001 0 0) 0x7f0e5c0035b0 con 0x7f0e6c0b36f0 2023-07-06T01:21:51.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 msgr2=0x7f0e6c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:51.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 0x7f0e6c0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:51.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 >> v1:172.21.15.103:6789/0 conn(0x7f0e6c0b3b70 legacy=0x7f0e6c0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:51.811 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c0ce2a0 con 0x7f0e6c0b36f0 2023-07-06T01:21:51.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f0e60001670 con 0x7f0e6c0b36f0 2023-07-06T01:21:51.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 >> v1:172.21.15.22:6789/0 conn(0x7f0e6c0b36f0 legacy=0x7f0e6c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:51.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e5c003a30 0x7f0e5c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 0x7f0e5c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.812 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e5c004d90 0x7f0e5c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c15cff0 con 0x7f0e5c004d90 2023-07-06T01:21:51.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c00bb20 con 0x7f0e5c003a30 2023-07-06T01:21:51.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.804+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c00bce0 con 0x7f0e6c0d11a0 2023-07-06T01:21:51.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e70de2700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e5c003a30 0x7f0e5c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.813 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e6bfff700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 0x7f0e5c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e715e3700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e5c004d90 0x7f0e5c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e6bfff700 1 -- 172.21.15.22:0/1446198090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e5c003a30 msgr2=0x7f0e5c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:51.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e6bfff700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e5c003a30 0x7f0e5c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:51.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e6bfff700 1 -- 172.21.15.22:0/1446198090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e5c004d90 msgr2=0x7f0e5c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:51.814 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e6bfff700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e5c004d90 0x7f0e5c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:51.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e6bfff700 1 -- 172.21.15.22:0/1446198090 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c0035b0 con 0x7f0e6c0d11a0 2023-07-06T01:21:51.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e715e3700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e5c004d90 0x7f0e5c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:51.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e70de2700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e5c003a30 0x7f0e5c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:51.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e6bfff700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 0x7f0e5c004850 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e580018b0 tx=0x7f0e5c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed45fc2e664a7676 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:51.815 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58001370 con 0x7f0e6c0d11a0 2023-07-06T01:21:51.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5800c6b0 con 0x7f0e6c0d11a0 2023-07-06T01:21:51.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.805+0000 7f0e6b7fe700 1 -- 172.21.15.22:0/1446198090 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800ca60 con 0x7f0e6c0d11a0 2023-07-06T01:21:51.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.806+0000 7f0e73046700 1 -- 172.21.15.22:0/1446198090 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 msgr2=0x7f0e5c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:51.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.806+0000 7f0e73046700 1 --2- 172.21.15.22:0/1446198090 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 0x7f0e5c004850 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e580018b0 tx=0x7f0e5c005880 comp rx=0 tx=0).stop 2023-07-06T01:21:51.816 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.806+0000 7f0e715e3700 1 -- 172.21.15.22:0/1446198090 reap_dead start 2023-07-06T01:21:51.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.806+0000 7f0e73046700 1 -- 172.21.15.22:0/1446198090 shutdown_connections 2023-07-06T01:21:51.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.806+0000 7f0e73046700 1 -- 172.21.15.22:0/1446198090 >> 172.21.15.22:0/1446198090 conn(0x7f0e6c09c2e0 msgr2=0x7f0e6c0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:51.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.806+0000 7f0e73046700 1 -- 172.21.15.22:0/1446198090 shutdown_connections 2023-07-06T01:21:51.817 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.806+0000 7f0e73046700 1 -- 172.21.15.22:0/1446198090 wait complete. 2023-07-06T01:21:51.819 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.808+0000 7f0e73046700 1 Processor -- start 2023-07-06T01:21:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.808+0000 7f0e73046700 1 -- start start 2023-07-06T01:21:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e73046700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 0x7f0e6c1e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e73046700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e6c1593d0 0x7f0e6c158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.820 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e73046700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e6c1e2a00 0x7f0e6c1e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e73046700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c1e33e0 con 0x7f0e6c1e2a00 2023-07-06T01:21:51.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e73046700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c1e7260 con 0x7f0e6c1593d0 2023-07-06T01:21:51.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e73046700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c15a580 con 0x7f0e6c0d11a0 2023-07-06T01:21:51.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e715e3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e6c1e2a00 0x7f0e6c1e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.821 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e715e3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e6c1e2a00 0x7f0e6c1e2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48218/0 (socket says 172.21.15.22:48218) 2023-07-06T01:21:51.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e715e3700 1 -- 172.21.15.22:0/4287851263 learned_addr learned my addr 172.21.15.22:0/4287851263 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:51.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e70de2700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 0x7f0e6c1e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e6bfff700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e6c1593d0 0x7f0e6c158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e715e3700 1 -- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 msgr2=0x7f0e6c1e6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:51.822 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e715e3700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 0x7f0e6c1e6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:51.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e715e3700 1 -- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e6c1593d0 msgr2=0x7f0e6c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:51.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e715e3700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e6c1593d0 0x7f0e6c158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:51.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e715e3700 1 -- 172.21.15.22:0/4287851263 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c159970 con 0x7f0e6c1e2a00 2023-07-06T01:21:51.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e6bfff700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e6c1593d0 0x7f0e6c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:51.823 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e715e3700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e6c1e2a00 0x7f0e6c1e2ea0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c08bbb0 tx=0x7f0e6400f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:51.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e697fa700 1 -- 172.21.15.22:0/4287851263 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6400e040 con 0x7f0e6c1e2a00 2023-07-06T01:21:51.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e697fa700 1 -- 172.21.15.22:0/4287851263 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6400f040 con 0x7f0e6c1e2a00 2023-07-06T01:21:51.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e697fa700 1 -- 172.21.15.22:0/4287851263 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6400fb90 con 0x7f0e6c1e2a00 2023-07-06T01:21:51.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.809+0000 7f0e73046700 1 -- 172.21.15.22:0/4287851263 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c159c50 con 0x7f0e6c1e2a00 2023-07-06T01:21:51.824 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.810+0000 7f0e73046700 1 -- 172.21.15.22:0/4287851263 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c15a210 con 0x7f0e6c1e2a00 2023-07-06T01:21:51.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.810+0000 7f0e73046700 1 -- 172.21.15.22:0/4287851263 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6c0b4840 con 0x7f0e6c1e2a00 2023-07-06T01:21:51.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.811+0000 7f0e697fa700 1 -- 172.21.15.22:0/4287851263 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e6400fcf0 con 0x7f0e6c1e2a00 2023-07-06T01:21:51.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.811+0000 7f0e697fa700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0e5006ad10 0x7f0e5006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:51.825 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.811+0000 7f0e697fa700 1 -- 172.21.15.22:0/4287851263 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0e64089650 con 0x7f0e6c1e2a00 2023-07-06T01:21:51.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.811+0000 7f0e70de2700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0e5006ad10 0x7f0e5006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:51.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.812+0000 7f0e70de2700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0e5006ad10 0x7f0e5006d1c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0e60001200 tx=0x7f0e60000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:51.826 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:51.816+0000 7f0e697fa700 1 -- 172.21.15.22:0/4287851263 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0e64054a90 con 0x7f0e6c1e2a00 2023-07-06T01:21:52.015 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.013+0000 7f0e73046700 1 -- 172.21.15.22:0/4287851263 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e6c0ce860 con 0x7f0e5006ad10 2023-07-06T01:21:52.016 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.014+0000 7f0e697fa700 1 -- 172.21.15.22:0/4287851263 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0e600070c0 con 0x7f0e5006ad10 2023-07-06T01:21:52.019 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.016+0000 7f0e73046700 1 -- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0e5006ad10 msgr2=0x7f0e5006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.016+0000 7f0e73046700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0e5006ad10 0x7f0e5006d1c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0e60001200 tx=0x7f0e60000ed0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.016+0000 7f0e73046700 1 -- 172.21.15.22:0/4287851263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e6c1e2a00 msgr2=0x7f0e6c1e2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.020 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.016+0000 7f0e73046700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e6c1e2a00 0x7f0e6c1e2ea0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c08bbb0 tx=0x7f0e6400f490 comp rx=0 tx=0).stop 2023-07-06T01:21:52.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.016+0000 7f0e73046700 1 -- 172.21.15.22:0/4287851263 shutdown_connections 2023-07-06T01:21:52.021 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.016+0000 7f0e73046700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e6c1e2a00 0x7f0e6c1e2ea0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.016+0000 7f0e73046700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0e6c0d11a0 0x7f0e6c1e6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.022 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.016+0000 7f0e73046700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f0e5006ad10 0x7f0e5006d1c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.016+0000 7f0e73046700 1 --2- 172.21.15.22:0/4287851263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e6c1593d0 0x7f0e6c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.023 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.016+0000 7f0e73046700 1 -- 172.21.15.22:0/4287851263 >> 172.21.15.22:0/4287851263 conn(0x7f0e6c09c2e0 msgr2=0x7f0e6c0cd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:52.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.016+0000 7f0e73046700 1 -- 172.21.15.22:0/4287851263 shutdown_connections 2023-07-06T01:21:52.024 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.016+0000 7f0e73046700 1 -- 172.21.15.22:0/4287851263 wait complete. 2023-07-06T01:21:52.024 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 0 to deep-scrub 2023-07-06T01:21:52.030 INFO:tasks.ceph:Scrubbing osd.1 2023-07-06T01:21:52.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-07-06T01:21:52.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.158+0000 7f2b9f2e4700 1 Processor -- start 2023-07-06T01:21:52.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.159+0000 7f2b9f2e4700 1 -- start start 2023-07-06T01:21:52.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.159+0000 7f2b9f2e4700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f2b980ccd70 0x7f2b9815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.160+0000 7f2b9f2e4700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b980cf230 con 0x7f2b98159360 2023-07-06T01:21:52.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.160+0000 7f2b9f2e4700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b980ce620 con 0x7f2b9806d040 2023-07-06T01:21:52.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.160+0000 7f2b9f2e4700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9815df20 con 0x7f2b980ccd70 2023-07-06T01:21:52.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.160+0000 7f2b9d080700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f2b98159360 0x7f2b980cc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38966/0 (socket says 172.21.15.22:38966) 2023-07-06T01:21:52.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.160+0000 7f2b9d080700 1 -- 172.21.15.22:0/3507293730 learned_addr learned my addr 172.21.15.22:0/3507293730 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:52.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.160+0000 7f2b9c87f700 1 --2- 172.21.15.22:0/3507293730 >> v2:172.21.15.103:3300/0 conn(0x7f2b980ccd70 0x7f2b9815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.160+0000 7f2b8ffff700 1 -- 172.21.15.22:0/3507293730 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914524584 0 0) 0x7f2b980cf230 con 0x7f2b98159360 2023-07-06T01:21:52.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.160+0000 7f2b8ffff700 1 -- 172.21.15.22:0/3507293730 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b840035b0 con 0x7f2b98159360 2023-07-06T01:21:52.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.160+0000 7f2b8ffff700 1 -- 172.21.15.22:0/3507293730 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 736741371 0 0) 0x7f2b840035b0 con 0x7f2b98159360 2023-07-06T01:21:52.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.160+0000 7f2b9c87f700 1 -- 172.21.15.22:0/3507293730 >> v1:172.21.15.103:6789/0 conn(0x7f2b9806d040 legacy=0x7f2b9806d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.160+0000 7f2b9c87f700 1 -- 172.21.15.22:0/3507293730 >> v1:172.21.15.22:6789/0 conn(0x7f2b98159360 legacy=0x7f2b980cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.160+0000 7f2b9c87f700 1 -- 172.21.15.22:0/3507293730 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b980ce900 con 0x7f2b980ccd70 2023-07-06T01:21:52.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.160+0000 7f2b9c87f700 1 --2- 172.21.15.22:0/3507293730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b980ccd70 0x7f2b9815d9e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009840 tx=0x7f2b94009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6511b53546da303c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.161+0000 7f2b8ffff700 1 -- 172.21.15.22:0/3507293730 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88001670 con 0x7f2b980ccd70 2023-07-06T01:21:52.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.161+0000 7f2b8ffff700 1 -- 172.21.15.22:0/3507293730 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b94004030 con 0x7f2b980ccd70 2023-07-06T01:21:52.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.161+0000 7f2b8ffff700 1 -- 172.21.15.22:0/3507293730 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b940043a0 con 0x7f2b980ccd70 2023-07-06T01:21:52.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.161+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3507293730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b980ccd70 msgr2=0x7f2b9815d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.161+0000 7f2b9f2e4700 1 --2- 172.21.15.22:0/3507293730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b980ccd70 0x7f2b9815d9e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009840 tx=0x7f2b94009b50 comp rx=0 tx=0).stop 2023-07-06T01:21:52.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.161+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3507293730 shutdown_connections 2023-07-06T01:21:52.167 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.161+0000 7f2b9f2e4700 1 --2- 172.21.15.22:0/3507293730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b980ccd70 0x7f2b9815d9e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.161+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3507293730 >> 172.21.15.22:0/3507293730 conn(0x7f2b980b1890 msgr2=0x7f2b980b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:52.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.161+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3507293730 shutdown_connections 2023-07-06T01:21:52.168 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.161+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3507293730 wait complete. 2023-07-06T01:21:52.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.162+0000 7f2b9f2e4700 1 Processor -- start 2023-07-06T01:21:52.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.162+0000 7f2b9f2e4700 1 -- start start 2023-07-06T01:21:52.171 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.163+0000 7f2b9f2e4700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b9806d040 0x7f2b98159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.163+0000 7f2b9f2e4700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b980ccd70 0x7f2b98142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.163+0000 7f2b9f2e4700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b98159360 0x7f2b98143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.163+0000 7f2b9f2e4700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b42c0 con 0x7f2b9806d040 2023-07-06T01:21:52.172 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.163+0000 7f2b9f2e4700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b981ef590 con 0x7f2b980ccd70 2023-07-06T01:21:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.163+0000 7f2b9f2e4700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b3640 con 0x7f2b98159360 2023-07-06T01:21:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.163+0000 7f2b9d080700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b9806d040 0x7f2b98159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.163+0000 7f2b9d080700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b9806d040 0x7f2b98159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48228/0 (socket says 172.21.15.22:48228) 2023-07-06T01:21:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.163+0000 7f2b9d080700 1 -- 172.21.15.22:0/3339745187 learned_addr learned my addr 172.21.15.22:0/3339745187 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:52.173 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.163+0000 7f2b9d881700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b98159360 0x7f2b98143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.163+0000 7f2b9c87f700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b980ccd70 0x7f2b98142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.164+0000 7f2b9d080700 1 -- 172.21.15.22:0/3339745187 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b98159360 msgr2=0x7f2b98143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.164+0000 7f2b9d080700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b98159360 0x7f2b98143390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.164+0000 7f2b9d080700 1 -- 172.21.15.22:0/3339745187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b980ccd70 msgr2=0x7f2b98142e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.164+0000 7f2b9d080700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b980ccd70 0x7f2b98142e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.174 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.164+0000 7f2b9d080700 1 -- 172.21.15.22:0/3339745187 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b940094a0 con 0x7f2b9806d040 2023-07-06T01:21:52.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.164+0000 7f2b9d080700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b9806d040 0x7f2b98159900 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800e940 tx=0x7f2b8800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.164+0000 7f2b8dffb700 1 -- 172.21.15.22:0/3339745187 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8800ee20 con 0x7f2b9806d040 2023-07-06T01:21:52.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.164+0000 7f2b8dffb700 1 -- 172.21.15.22:0/3339745187 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b88004a80 con 0x7f2b9806d040 2023-07-06T01:21:52.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.164+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3339745187 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b980b3920 con 0x7f2b9806d040 2023-07-06T01:21:52.175 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.164+0000 7f2b8dffb700 1 -- 172.21.15.22:0/3339745187 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88004e30 con 0x7f2b9806d040 2023-07-06T01:21:52.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.164+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3339745187 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b980b3ee0 con 0x7f2b9806d040 2023-07-06T01:21:52.176 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.165+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3339745187 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b980ceec0 con 0x7f2b9806d040 2023-07-06T01:21:52.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.165+0000 7f2b8dffb700 1 -- 172.21.15.22:0/3339745187 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2b880070d0 con 0x7f2b9806d040 2023-07-06T01:21:52.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.166+0000 7f2b8dffb700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2b7c06ace0 0x7f2b7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.166+0000 7f2b8dffb700 1 -- 172.21.15.22:0/3339745187 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2b8808b730 con 0x7f2b9806d040 2023-07-06T01:21:52.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.166+0000 7f2b9c87f700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2b7c06ace0 0x7f2b7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.177 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.166+0000 7f2b8dffb700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] conn(0x7f2b7c071150 0x7f2b7c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.166+0000 7f2b8dffb700 1 -- 172.21.15.22:0/3339745187 --> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b7c073c30 con 0x7f2b7c071150 2023-07-06T01:21:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.166+0000 7f2b8dffb700 1 -- 172.21.15.22:0/3339745187 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b8808ddd0 con 0x7f2b9806d040 2023-07-06T01:21:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.166+0000 7f2b9d881700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] conn(0x7f2b7c071150 0x7f2b7c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.167+0000 7f2b9d881700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] conn(0x7f2b7c071150 0x7f2b7c073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.178 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.167+0000 7f2b9c87f700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2b7c06ace0 0x7f2b7c06d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2b980cd1b0 tx=0x7f2b94008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.167+0000 7f2b8dffb700 1 -- 172.21.15.22:0/3339745187 <== osd.1 v2:172.21.15.22:6800/2676174112 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2b7c073c30 con 0x7f2b7c071150 2023-07-06T01:21:52.183 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.182+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3339745187 --> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2b980cebe0 con 0x7f2b7c071150 2023-07-06T01:21:52.184 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.182+0000 7f2b8dffb700 1 -- 172.21.15.22:0/3339745187 <== osd.1 v2:172.21.15.22:6800/2676174112 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2b90003a00 con 0x7f2b7c071150 2023-07-06T01:21:52.184 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:21:52.184 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T01:21:52.185 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:21:52.185 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.182+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3339745187 >> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] conn(0x7f2b7c071150 msgr2=0x7f2b7c073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.182+0000 7f2b9f2e4700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.22:6800/2676174112,v1:172.21.15.22:6801/2676174112] conn(0x7f2b7c071150 0x7f2b7c073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.182+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3339745187 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2b7c06ace0 msgr2=0x7f2b7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.182+0000 7f2b9f2e4700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f2b7c06ace0 0x7f2b7c06d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2b980cd1b0 tx=0x7f2b94008030 comp rx=0 tx=0).stop 2023-07-06T01:21:52.186 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.183+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3339745187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b9806d040 msgr2=0x7f2b98159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.183+0000 7f2b9f2e4700 1 --2- 172.21.15.22:0/3339745187 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b9806d040 0x7f2b98159900 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800e940 tx=0x7f2b8800ed00 comp rx=0 tx=0).stop 2023-07-06T01:21:52.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.183+0000 7f2b9d881700 1 -- 172.21.15.22:0/3339745187 reap_dead start 2023-07-06T01:21:52.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.183+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3339745187 shutdown_connections 2023-07-06T01:21:52.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.183+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3339745187 >> 172.21.15.22:0/3339745187 conn(0x7f2b980b1890 msgr2=0x7f2b980b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:52.187 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.183+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3339745187 shutdown_connections 2023-07-06T01:21:52.188 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.183+0000 7f2b9f2e4700 1 -- 172.21.15.22:0/3339745187 wait complete. 2023-07-06T01:21:52.196 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-07-06T01:21:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.324+0000 7f7c923cb700 1 Processor -- start 2023-07-06T01:21:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.324+0000 7f7c923cb700 1 -- start start 2023-07-06T01:21:52.327 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.324+0000 7f7c923cb700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7c8c159360 0x7f7c8c15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.324+0000 7f7c923cb700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7c8c0ccae0 0x7f7c8c071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.324+0000 7f7c923cb700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8c0cf230 con 0x7f7c8c0cc660 2023-07-06T01:21:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.324+0000 7f7c923cb700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8c15df20 con 0x7f7c8c0ccae0 2023-07-06T01:21:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.324+0000 7f7c923cb700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8c0ce620 con 0x7f7c8c159360 2023-07-06T01:21:52.328 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.325+0000 7f7c90968700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7c8c0ccae0 0x7f7c8c071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.325+0000 7f7c90968700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7c8c0ccae0 0x7f7c8c071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48242/0 (socket says 172.21.15.22:48242) 2023-07-06T01:21:52.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.325+0000 7f7c90968700 1 -- 172.21.15.22:0/661503178 learned_addr learned my addr 172.21.15.22:0/661503178 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:52.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.325+0000 7f7c8bfff700 1 --2- 172.21.15.22:0/661503178 >> v2:172.21.15.103:3300/0 conn(0x7f7c8c159360 0x7f7c8c15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.325+0000 7f7c90968700 1 -- 172.21.15.22:0/661503178 >> v2:172.21.15.103:3300/0 conn(0x7f7c8c159360 msgr2=0x7f7c8c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:52.329 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.325+0000 7f7c90968700 1 --2- 172.21.15.22:0/661503178 >> v2:172.21.15.103:3300/0 conn(0x7f7c8c159360 0x7f7c8c15d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.325+0000 7f7c90968700 1 -- 172.21.15.22:0/661503178 >> v1:172.21.15.103:6789/0 conn(0x7f7c8c0cc660 legacy=0x7f7c8c070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.325+0000 7f7c90968700 1 -- 172.21.15.22:0/661503178 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8c0ce900 con 0x7f7c8c0ccae0 2023-07-06T01:21:52.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.325+0000 7f7c90968700 1 --2- 172.21.15.22:0/661503178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8c0ccae0 0x7f7c8c071130 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c071aa0 tx=0x7f7c8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798c82c6098ae428 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.325+0000 7f7c8affd700 1 -- 172.21.15.22:0/661503178 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8400e070 con 0x7f7c8c0ccae0 2023-07-06T01:21:52.330 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.325+0000 7f7c8affd700 1 -- 172.21.15.22:0/661503178 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8400b9c0 con 0x7f7c8c0ccae0 2023-07-06T01:21:52.331 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.325+0000 7f7c8affd700 1 -- 172.21.15.22:0/661503178 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8400bd70 con 0x7f7c8c0ccae0 2023-07-06T01:21:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.326+0000 7f7c923cb700 1 -- 172.21.15.22:0/661503178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8c0ccae0 msgr2=0x7f7c8c071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.334 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.326+0000 7f7c923cb700 1 --2- 172.21.15.22:0/661503178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8c0ccae0 0x7f7c8c071130 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c071aa0 tx=0x7f7c8400b330 comp rx=0 tx=0).stop 2023-07-06T01:21:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.326+0000 7f7c923cb700 1 -- 172.21.15.22:0/661503178 shutdown_connections 2023-07-06T01:21:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.326+0000 7f7c923cb700 1 --2- 172.21.15.22:0/661503178 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8c0ccae0 0x7f7c8c071130 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.326+0000 7f7c923cb700 1 --2- 172.21.15.22:0/661503178 >> v2:172.21.15.103:3300/0 conn(0x7f7c8c159360 0x7f7c8c15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.335 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.326+0000 7f7c923cb700 1 -- 172.21.15.22:0/661503178 >> 172.21.15.22:0/661503178 conn(0x7f7c8c0b1890 msgr2=0x7f7c8c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.326+0000 7f7c923cb700 1 -- 172.21.15.22:0/661503178 shutdown_connections 2023-07-06T01:21:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.326+0000 7f7c923cb700 1 -- 172.21.15.22:0/661503178 wait complete. 2023-07-06T01:21:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.327+0000 7f7c923cb700 1 Processor -- start 2023-07-06T01:21:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.327+0000 7f7c923cb700 1 -- start start 2023-07-06T01:21:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c923cb700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8c0cc660 0x7f7c8c1ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c923cb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c8c0ccae0 0x7f7c8c0c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.337 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c923cb700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c8c159360 0x7f7c8c0b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c923cb700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c140ca0 con 0x7f7c8c0cc660 2023-07-06T01:21:52.338 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c923cb700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c1ef920 con 0x7f7c8c159360 2023-07-06T01:21:52.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c923cb700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c0c9c30 con 0x7f7c8c0ccae0 2023-07-06T01:21:52.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c8bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8c0cc660 0x7f7c8c1ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.339 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c8bfff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8c0cc660 0x7f7c8c1ef3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48244/0 (socket says 172.21.15.22:48244) 2023-07-06T01:21:52.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c8bfff700 1 -- 172.21.15.22:0/3080603459 learned_addr learned my addr 172.21.15.22:0/3080603459 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:52.340 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c90968700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c8c159360 0x7f7c8c0b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c8b7fe700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c8c0ccae0 0x7f7c8c0c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c8bfff700 1 -- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c8c0ccae0 msgr2=0x7f7c8c0c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c8bfff700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c8c0ccae0 0x7f7c8c0c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c8bfff700 1 -- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c8c159360 msgr2=0x7f7c8c0b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.342 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c8bfff700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c8c159360 0x7f7c8c0b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c8bfff700 1 -- 172.21.15.22:0/3080603459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c84009b30 con 0x7f7c8c0cc660 2023-07-06T01:21:52.343 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c90968700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c8c159360 0x7f7c8c0b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:52.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.328+0000 7f7c8b7fe700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c8c0ccae0 0x7f7c8c0c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:52.344 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.329+0000 7f7c8bfff700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8c0cc660 0x7f7c8c1ef3e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00d800 tx=0x7f7c7c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.329+0000 7f7c88ff9700 1 -- 172.21.15.22:0/3080603459 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c0125f0 con 0x7f7c8c0cc660 2023-07-06T01:21:52.345 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.329+0000 7f7c88ff9700 1 -- 172.21.15.22:0/3080603459 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7c012750 con 0x7f7c8c0cc660 2023-07-06T01:21:52.346 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.329+0000 7f7c923cb700 1 -- 172.21.15.22:0/3080603459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8c140f60 con 0x7f7c8c0cc660 2023-07-06T01:21:52.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.329+0000 7f7c88ff9700 1 -- 172.21.15.22:0/3080603459 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c012b00 con 0x7f7c8c0cc660 2023-07-06T01:21:52.347 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.329+0000 7f7c923cb700 1 -- 172.21.15.22:0/3080603459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8c143960 con 0x7f7c8c0cc660 2023-07-06T01:21:52.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.330+0000 7f7c923cb700 1 -- 172.21.15.22:0/3080603459 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8c0744b0 con 0x7f7c8c0cc660 2023-07-06T01:21:52.348 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.330+0000 7f7c88ff9700 1 -- 172.21.15.22:0/3080603459 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7c7c012c60 con 0x7f7c8c0cc660 2023-07-06T01:21:52.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.331+0000 7f7c88ff9700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7c7806ace0 0x7f7c7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.349 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.331+0000 7f7c88ff9700 1 -- 172.21.15.22:0/3080603459 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7c7c089540 con 0x7f7c8c0cc660 2023-07-06T01:21:52.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.331+0000 7f7c8b7fe700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7c7806ace0 0x7f7c7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.350 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.331+0000 7f7c8b7fe700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7c7806ace0 0x7f7c7806d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c15ca40 tx=0x7f7c80009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.335+0000 7f7c88ff9700 1 -- 172.21.15.22:0/3080603459 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7c7c054980 con 0x7f7c8c0cc660 2023-07-06T01:21:52.534 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.533+0000 7f7c923cb700 1 -- 172.21.15.22:0/3080603459 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c8c140de0 con 0x7f7c7806ace0 2023-07-06T01:21:52.535 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.533+0000 7f7c88ff9700 1 -- 172.21.15.22:0/3080603459 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7c80001c40 con 0x7f7c7806ace0 2023-07-06T01:21:52.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.535+0000 7f7c923cb700 1 -- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7c7806ace0 msgr2=0x7f7c7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.537 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.535+0000 7f7c923cb700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7c7806ace0 0x7f7c7806d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c15ca40 tx=0x7f7c80009040 comp rx=0 tx=0).stop 2023-07-06T01:21:52.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.535+0000 7f7c923cb700 1 -- 172.21.15.22:0/3080603459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8c0cc660 msgr2=0x7f7c8c1ef3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.535+0000 7f7c923cb700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8c0cc660 0x7f7c8c1ef3e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00d800 tx=0x7f7c7c00dbc0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.535+0000 7f7c923cb700 1 -- 172.21.15.22:0/3080603459 shutdown_connections 2023-07-06T01:21:52.538 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.535+0000 7f7c923cb700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7c7806ace0 0x7f7c7806d190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.535+0000 7f7c923cb700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c8c159360 0x7f7c8c0b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.535+0000 7f7c923cb700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c8c0cc660 0x7f7c8c1ef3e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.535+0000 7f7c923cb700 1 --2- 172.21.15.22:0/3080603459 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c8c0ccae0 0x7f7c8c0c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.535+0000 7f7c923cb700 1 -- 172.21.15.22:0/3080603459 >> 172.21.15.22:0/3080603459 conn(0x7f7c8c0b1890 msgr2=0x7f7c8c0ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:52.539 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.535+0000 7f7c923cb700 1 -- 172.21.15.22:0/3080603459 shutdown_connections 2023-07-06T01:21:52.540 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.535+0000 7f7c923cb700 1 -- 172.21.15.22:0/3080603459 wait complete. 2023-07-06T01:21:52.540 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 1 to deep-scrub 2023-07-06T01:21:52.549 INFO:tasks.ceph:Scrubbing osd.2 2023-07-06T01:21:52.550 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-07-06T01:21:52.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.679+0000 7f10716c5700 1 Processor -- start 2023-07-06T01:21:52.681 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.679+0000 7f10716c5700 1 -- start start 2023-07-06T01:21:52.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.680+0000 7f10716c5700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f106c06a710 0x7f106c06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.680+0000 7f10716c5700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f106c0cbc10 0x7f106c0cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.683 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.680+0000 7f10716c5700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106c0af670 con 0x7f106c0af2a0 2023-07-06T01:21:52.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.680+0000 7f10716c5700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106c0cd560 con 0x7f106c06a710 2023-07-06T01:21:52.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.680+0000 7f10716c5700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c15cab0 con 0x7f106c0cbc10 2023-07-06T01:21:52.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f106b7fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f106c0cbc10 0x7f106c0cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f106a7fc700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f106c06a710 0x7f106c06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.684 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f106affd700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f106c0af2a0 0x7f106c06a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:41060/0 (socket says 172.21.15.22:41060) 2023-07-06T01:21:52.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f106affd700 1 -- 172.21.15.22:0/3375002716 learned_addr learned my addr 172.21.15.22:0/3375002716 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:52.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f106a7fc700 1 -- 172.21.15.22:0/3375002716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f106c0cbc10 msgr2=0x7f106c0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:52.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f106a7fc700 1 --2- 172.21.15.22:0/3375002716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f106c0cbc10 0x7f106c0cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f106a7fc700 1 -- 172.21.15.22:0/3375002716 >> v1:172.21.15.103:6789/0 conn(0x7f106c0af2a0 legacy=0x7f106c06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.685 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f106a7fc700 1 -- 172.21.15.22:0/3375002716 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c06d030 con 0x7f106c06a710 2023-07-06T01:21:52.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f1069ffb700 1 -- 172.21.15.22:0/3375002716 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237688130 0 0) 0x7f106c0af670 con 0x7f106c0af2a0 2023-07-06T01:21:52.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f106b7fe700 1 --2- 172.21.15.22:0/3375002716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f106c0cbc10 0x7f106c0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:52.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f106a7fc700 1 --2- 172.21.15.22:0/3375002716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f106c06a710 0x7f106c06caf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1060009840 tx=0x7f1060009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6f687d06ac5ac39 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f1069ffb700 1 -- 172.21.15.22:0/3375002716 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060011830 con 0x7f106c06a710 2023-07-06T01:21:52.686 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f1069ffb700 1 -- 172.21.15.22:0/3375002716 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1060011990 con 0x7f106c06a710 2023-07-06T01:21:52.687 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.681+0000 7f1069ffb700 1 -- 172.21.15.22:0/3375002716 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060011d20 con 0x7f106c06a710 2023-07-06T01:21:52.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.682+0000 7f10716c5700 1 -- 172.21.15.22:0/3375002716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f106c06a710 msgr2=0x7f106c06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.689 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.682+0000 7f10716c5700 1 --2- 172.21.15.22:0/3375002716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f106c06a710 0x7f106c06caf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1060009840 tx=0x7f1060009b50 comp rx=0 tx=0).stop 2023-07-06T01:21:52.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.682+0000 7f10716c5700 1 -- 172.21.15.22:0/3375002716 shutdown_connections 2023-07-06T01:21:52.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.682+0000 7f10716c5700 1 --2- 172.21.15.22:0/3375002716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f106c0cbc10 0x7f106c0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.682+0000 7f10716c5700 1 --2- 172.21.15.22:0/3375002716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f106c06a710 0x7f106c06caf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.682+0000 7f10716c5700 1 -- 172.21.15.22:0/3375002716 >> 172.21.15.22:0/3375002716 conn(0x7f106c0b1890 msgr2=0x7f106c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:52.690 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.682+0000 7f10716c5700 1 -- 172.21.15.22:0/3375002716 shutdown_connections 2023-07-06T01:21:52.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.682+0000 7f10716c5700 1 -- 172.21.15.22:0/3375002716 wait complete. 2023-07-06T01:21:52.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.683+0000 7f10716c5700 1 Processor -- start 2023-07-06T01:21:52.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.683+0000 7f10716c5700 1 -- start start 2023-07-06T01:21:52.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f10716c5700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f106c06a710 0x7f106c14aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.691 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f10716c5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f106c0af2a0 0x7f106c15e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f10716c5700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f106c0cbc10 0x7f106c15e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f10716c5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c1f6640 con 0x7f106c0af2a0 2023-07-06T01:21:52.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f10716c5700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106c14b510 con 0x7f106c0cbc10 2023-07-06T01:21:52.692 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f10716c5700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c14b1e0 con 0x7f106c06a710 2023-07-06T01:21:52.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f106a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f106c0af2a0 0x7f106c15e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f106a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f106c0af2a0 0x7f106c15e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48260/0 (socket says 172.21.15.22:48260) 2023-07-06T01:21:52.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f106a7fc700 1 -- 172.21.15.22:0/2287681072 learned_addr learned my addr 172.21.15.22:0/2287681072 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:52.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f106affd700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f106c06a710 0x7f106c14aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.693 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f106b7fe700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f106c0cbc10 0x7f106c15e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f106a7fc700 1 -- 172.21.15.22:0/2287681072 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f106c06a710 msgr2=0x7f106c14aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f106a7fc700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f106c06a710 0x7f106c14aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f106a7fc700 1 -- 172.21.15.22:0/2287681072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f106c0cbc10 msgr2=0x7f106c15e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.694 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f106a7fc700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f106c0cbc10 0x7f106c15e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.684+0000 7f106a7fc700 1 -- 172.21.15.22:0/2287681072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1064009a70 con 0x7f106c0af2a0 2023-07-06T01:21:52.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.685+0000 7f106a7fc700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f106c0af2a0 0x7f106c15e220 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1060009fd0 tx=0x7f1060004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.685+0000 7f10537fe700 1 -- 172.21.15.22:0/2287681072 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600045d0 con 0x7f106c0af2a0 2023-07-06T01:21:52.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.685+0000 7f10537fe700 1 -- 172.21.15.22:0/2287681072 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106001f070 con 0x7f106c0af2a0 2023-07-06T01:21:52.695 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.685+0000 7f10716c5700 1 -- 172.21.15.22:0/2287681072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10600094a0 con 0x7f106c0af2a0 2023-07-06T01:21:52.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.685+0000 7f10537fe700 1 -- 172.21.15.22:0/2287681072 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600038a0 con 0x7f106c0af2a0 2023-07-06T01:21:52.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.685+0000 7f10716c5700 1 -- 172.21.15.22:0/2287681072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106c1f5b60 con 0x7f106c0af2a0 2023-07-06T01:21:52.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.686+0000 7f10716c5700 1 -- 172.21.15.22:0/2287681072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f106c0cccc0 con 0x7f106c0af2a0 2023-07-06T01:21:52.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.686+0000 7f10537fe700 1 -- 172.21.15.22:0/2287681072 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1060003a00 con 0x7f106c0af2a0 2023-07-06T01:21:52.696 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.687+0000 7f10537fe700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f105806ace0 0x7f105806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.687+0000 7f10537fe700 1 -- 172.21.15.22:0/2287681072 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1060021070 con 0x7f106c0af2a0 2023-07-06T01:21:52.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.687+0000 7f10537fe700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] conn(0x7f1058071150 0x7f1058073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.687+0000 7f10537fe700 1 -- 172.21.15.22:0/2287681072 --> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1058073c30 con 0x7f1058071150 2023-07-06T01:21:52.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.687+0000 7f10537fe700 1 -- 172.21.15.22:0/2287681072 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f106008f2a0 con 0x7f106c0af2a0 2023-07-06T01:21:52.697 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.687+0000 7f106affd700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f105806ace0 0x7f105806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.687+0000 7f106b7fe700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] conn(0x7f1058071150 0x7f1058073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.687+0000 7f106affd700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f105806ace0 0x7f105806d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f105c007fd0 tx=0x7f105c007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.687+0000 7f106b7fe700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] conn(0x7f1058071150 0x7f1058073560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.698 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.688+0000 7f10537fe700 1 -- 172.21.15.22:0/2287681072 <== osd.2 v2:172.21.15.22:6816/579920713 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1058073c30 con 0x7f1058071150 2023-07-06T01:21:52.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.702+0000 7f10716c5700 1 -- 172.21.15.22:0/2287681072 --> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f106c0cc9e0 con 0x7f1058071150 2023-07-06T01:21:52.704 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.703+0000 7f10537fe700 1 -- 172.21.15.22:0/2287681072 <== osd.2 v2:172.21.15.22:6816/579920713 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f106400f040 con 0x7f1058071150 2023-07-06T01:21:52.705 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:21:52.705 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T01:21:52.705 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:21:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.703+0000 7f10716c5700 1 -- 172.21.15.22:0/2287681072 >> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] conn(0x7f1058071150 msgr2=0x7f1058073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.706 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.703+0000 7f10716c5700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.22:6816/579920713,v1:172.21.15.22:6817/579920713] conn(0x7f1058071150 0x7f1058073560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.703+0000 7f10716c5700 1 -- 172.21.15.22:0/2287681072 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f105806ace0 msgr2=0x7f105806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.703+0000 7f10716c5700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f105806ace0 0x7f105806d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f105c007fd0 tx=0x7f105c007de0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.703+0000 7f10716c5700 1 -- 172.21.15.22:0/2287681072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f106c0af2a0 msgr2=0x7f106c15e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.703+0000 7f10716c5700 1 --2- 172.21.15.22:0/2287681072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f106c0af2a0 0x7f106c15e220 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1060009fd0 tx=0x7f1060004750 comp rx=0 tx=0).stop 2023-07-06T01:21:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.703+0000 7f106b7fe700 1 -- 172.21.15.22:0/2287681072 reap_dead start 2023-07-06T01:21:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.703+0000 7f10716c5700 1 -- 172.21.15.22:0/2287681072 shutdown_connections 2023-07-06T01:21:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.703+0000 7f10716c5700 1 -- 172.21.15.22:0/2287681072 >> 172.21.15.22:0/2287681072 conn(0x7f106c0b1890 msgr2=0x7f106c06d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.703+0000 7f10716c5700 1 -- 172.21.15.22:0/2287681072 shutdown_connections 2023-07-06T01:21:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.703+0000 7f10716c5700 1 -- 172.21.15.22:0/2287681072 wait complete. 2023-07-06T01:21:52.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-07-06T01:21:52.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.840+0000 7f4ef67d8700 1 Processor -- start 2023-07-06T01:21:52.842 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.840+0000 7f4ef67d8700 1 -- start start 2023-07-06T01:21:52.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.841+0000 7f4ef67d8700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4ef0071980 0x7f4ef015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.841+0000 7f4ef67d8700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef015f0b0 con 0x7f4ef015a6c0 2023-07-06T01:21:52.845 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.841+0000 7f4ef67d8700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef015aa90 con 0x7f4ef006a710 2023-07-06T01:21:52.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.841+0000 7f4ef67d8700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef006f260 con 0x7f4ef0071980 2023-07-06T01:21:52.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.841+0000 7f4eeffff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f4ef015a6c0 0x7f4ef006a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:39012/0 (socket says 172.21.15.22:39012) 2023-07-06T01:21:52.846 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.841+0000 7f4eeffff700 1 -- 172.21.15.22:0/1747889016 learned_addr learned my addr 172.21.15.22:0/1747889016 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:52.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823588742 0 0) 0x7f4ee0001110 con 0x7f4ef015a6c0 2023-07-06T01:21:52.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eef7fe700 1 --2- 172.21.15.22:0/1747889016 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4ef0071980 0x7f4ef015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed80035b0 con 0x7f4ef015a6c0 2023-07-06T01:21:52.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1890475518 0 0) 0x7f4ed80035b0 con 0x7f4ef015a6c0 2023-07-06T01:21:52.847 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4ef0071980 msgr2=0x7f4ef015eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:52.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 --2- 172.21.15.22:0/1747889016 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4ef0071980 0x7f4ef015eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 >> v1:172.21.15.103:6789/0 conn(0x7f4ef006a710 legacy=0x7f4ef00cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef006e5e0 con 0x7f4ef015a6c0 2023-07-06T01:21:52.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4126994148 0 0) 0x7f4ee0001800 con 0x7f4ef015a6c0 2023-07-06T01:21:52.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 >> v1:172.21.15.22:6789/0 conn(0x7f4ef015a6c0 legacy=0x7f4ef006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 --2- 172.21.15.22:0/1747889016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ed8003a30 0x7f4ed8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 --2- 172.21.15.22:0/1747889016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ed8004850 0x7f4ed8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 --2- 172.21.15.22:0/1747889016 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4ed8005200 0x7f4ed800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef006f260 con 0x7f4ed8004850 2023-07-06T01:21:52.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed800bf50 con 0x7f4ed8003a30 2023-07-06T01:21:52.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed800c110 con 0x7f4ed8005200 2023-07-06T01:21:52.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eef7fe700 1 --2- 172.21.15.22:0/1747889016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ed8004850 0x7f4ed8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.850 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eeffff700 1 --2- 172.21.15.22:0/1747889016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ed8003a30 0x7f4ed8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4ef4d75700 1 --2- 172.21.15.22:0/1747889016 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4ed8005200 0x7f4ed800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eef7fe700 1 -- 172.21.15.22:0/1747889016 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4ed8005200 msgr2=0x7f4ed800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:52.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.842+0000 7f4eef7fe700 1 --2- 172.21.15.22:0/1747889016 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4ed8005200 0x7f4ed800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4eef7fe700 1 -- 172.21.15.22:0/1747889016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ed8003a30 msgr2=0x7f4ed8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:52.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4eef7fe700 1 --2- 172.21.15.22:0/1747889016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ed8003a30 0x7f4ed8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4eef7fe700 1 -- 172.21.15.22:0/1747889016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed80035b0 con 0x7f4ed8004850 2023-07-06T01:21:52.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4eef7fe700 1 --2- 172.21.15.22:0/1747889016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ed8004850 0x7f4ed8004cc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009b30 tx=0x7f4ed8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649dcfaad6e3f63c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400e6c0 con 0x7f4ed8004850 2023-07-06T01:21:52.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee400e820 con 0x7f4ed8004850 2023-07-06T01:21:52.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4eeeffd700 1 -- 172.21.15.22:0/1747889016 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400eb90 con 0x7f4ed8004850 2023-07-06T01:21:52.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1747889016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ed8004850 msgr2=0x7f4ed8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4ef67d8700 1 --2- 172.21.15.22:0/1747889016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ed8004850 0x7f4ed8004cc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009b30 tx=0x7f4ed8005880 comp rx=0 tx=0).stop 2023-07-06T01:21:52.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4ef4d75700 1 -- 172.21.15.22:0/1747889016 reap_dead start 2023-07-06T01:21:52.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1747889016 shutdown_connections 2023-07-06T01:21:52.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1747889016 >> 172.21.15.22:0/1747889016 conn(0x7f4ef00b1890 msgr2=0x7f4ef00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:52.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1747889016 shutdown_connections 2023-07-06T01:21:52.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.843+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1747889016 wait complete. 2023-07-06T01:21:52.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.845+0000 7f4ef67d8700 1 Processor -- start 2023-07-06T01:21:52.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.845+0000 7f4ef67d8700 1 -- start start 2023-07-06T01:21:52.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4ef67d8700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4ed8004850 0x7f4ef0153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4ef67d8700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ef015bc90 0x7f4ef01588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4ef67d8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef00b2b90 0x7f4ef00cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4ef67d8700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef0156e40 con 0x7f4ef015bc90 2023-07-06T01:21:52.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4ef67d8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef0154500 con 0x7f4ef00b2b90 2023-07-06T01:21:52.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4ef67d8700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef015c1c0 con 0x7f4ed8004850 2023-07-06T01:21:52.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4eef7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ef015bc90 0x7f4ef01588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4eef7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ef015bc90 0x7f4ef01588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48282/0 (socket says 172.21.15.22:48282) 2023-07-06T01:21:52.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4eef7fe700 1 -- 172.21.15.22:0/1207331749 learned_addr learned my addr 172.21.15.22:0/1207331749 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:52.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4eeffff700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4ed8004850 0x7f4ef0153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4ef4d75700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef00b2b90 0x7f4ef00cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4eef7fe700 1 -- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4ed8004850 msgr2=0x7f4ef0153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4eef7fe700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4ed8004850 0x7f4ef0153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.859 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4eef7fe700 1 -- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef00b2b90 msgr2=0x7f4ef00cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:52.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4eef7fe700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef00b2b90 0x7f4ef00cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:52.860 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4eef7fe700 1 -- 172.21.15.22:0/1207331749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef015c4a0 con 0x7f4ef015bc90 2023-07-06T01:21:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4ef4d75700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef00b2b90 0x7f4ef00cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4eef7fe700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ef015bc90 0x7f4ef01588d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8005810 tx=0x7f4ee40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4eecff9700 1 -- 172.21.15.22:0/1207331749 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400e6c0 con 0x7f4ef015bc90 2023-07-06T01:21:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4eecff9700 1 -- 172.21.15.22:0/1207331749 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee4013e70 con 0x7f4ef015bc90 2023-07-06T01:21:52.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4eecff9700 1 -- 172.21.15.22:0/1207331749 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4028670 con 0x7f4ef015bc90 2023-07-06T01:21:52.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.846+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1207331749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef015c780 con 0x7f4ef015bc90 2023-07-06T01:21:52.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.847+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1207331749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef0158e10 con 0x7f4ef015bc90 2023-07-06T01:21:52.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.847+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1207331749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef0156fb0 con 0x7f4ef015bc90 2023-07-06T01:21:52.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.848+0000 7f4eecff9700 1 -- 172.21.15.22:0/1207331749 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ee401c450 con 0x7f4ef015bc90 2023-07-06T01:21:52.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.848+0000 7f4eecff9700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4edc06ace0 0x7f4edc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:52.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.848+0000 7f4eecff9700 1 -- 172.21.15.22:0/1207331749 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4ee408ac10 con 0x7f4ef015bc90 2023-07-06T01:21:52.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.849+0000 7f4eeffff700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4edc06ace0 0x7f4edc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:52.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.849+0000 7f4eeffff700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4edc06ace0 0x7f4edc06d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0001070 tx=0x7f4ee0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:52.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:52.853+0000 7f4eecff9700 1 -- 172.21.15.22:0/1207331749 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4ef0156fb0 con 0x7f4ef015bc90 2023-07-06T01:21:53.054 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.052+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1207331749 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ef006eba0 con 0x7f4edc06ace0 2023-07-06T01:21:53.055 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.053+0000 7f4eecff9700 1 -- 172.21.15.22:0/1207331749 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4ee0007e40 con 0x7f4edc06ace0 2023-07-06T01:21:53.057 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.055+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4edc06ace0 msgr2=0x7f4edc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.058 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.055+0000 7f4ef67d8700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4edc06ace0 0x7f4edc06d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0001070 tx=0x7f4ee0009040 comp rx=0 tx=0).stop 2023-07-06T01:21:53.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.055+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1207331749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ef015bc90 msgr2=0x7f4ef01588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.059 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.055+0000 7f4ef67d8700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ef015bc90 0x7f4ef01588d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8005810 tx=0x7f4ee40047c0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.055+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1207331749 shutdown_connections 2023-07-06T01:21:53.060 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.055+0000 7f4ef67d8700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f4edc06ace0 0x7f4edc06d190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.055+0000 7f4ef67d8700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ef00b2b90 0x7f4ef00cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.061 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.055+0000 7f4ef67d8700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4ed8004850 0x7f4ef0153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.055+0000 7f4ef67d8700 1 --2- 172.21.15.22:0/1207331749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ef015bc90 0x7f4ef01588d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.062 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.055+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1207331749 >> 172.21.15.22:0/1207331749 conn(0x7f4ef00b1890 msgr2=0x7f4ef00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:53.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.055+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1207331749 shutdown_connections 2023-07-06T01:21:53.063 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.055+0000 7f4ef67d8700 1 -- 172.21.15.22:0/1207331749 wait complete. 2023-07-06T01:21:53.063 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 2 to deep-scrub 2023-07-06T01:21:53.070 INFO:tasks.ceph:Scrubbing osd.3 2023-07-06T01:21:53.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-07-06T01:21:53.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.199+0000 7fbdc22d5700 1 Processor -- start 2023-07-06T01:21:53.201 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.199+0000 7fbdc22d5700 1 -- start start 2023-07-06T01:21:53.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdc22d5700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbdbc06a710 0x7fbdbc06caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdc22d5700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fbdbc0cbc10 0x7fbdbc0cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdc22d5700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdbc0af670 con 0x7fbdbc0af2a0 2023-07-06T01:21:53.203 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdc22d5700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdbc0cd560 con 0x7fbdbc06a710 2023-07-06T01:21:53.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdc22d5700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdbc15cab0 con 0x7fbdbc0cbc10 2023-07-06T01:21:53.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdbb7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbdbc06a710 0x7fbdbc06caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdbb7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fbdbc06a710 0x7fbdbc06caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48288/0 (socket says 172.21.15.22:48288) 2023-07-06T01:21:53.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdbb7fe700 1 -- 172.21.15.22:0/1314441450 learned_addr learned my addr 172.21.15.22:0/1314441450 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:53.204 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdc0872700 1 --2- 172.21.15.22:0/1314441450 >> v2:172.21.15.103:3300/0 conn(0x7fbdbc0cbc10 0x7fbdbc0cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdbb7fe700 1 -- 172.21.15.22:0/1314441450 >> v2:172.21.15.103:3300/0 conn(0x7fbdbc0cbc10 msgr2=0x7fbdbc0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdbb7fe700 1 --2- 172.21.15.22:0/1314441450 >> v2:172.21.15.103:3300/0 conn(0x7fbdbc0cbc10 0x7fbdbc0cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdbb7fe700 1 -- 172.21.15.22:0/1314441450 >> v1:172.21.15.103:6789/0 conn(0x7fbdbc0af2a0 legacy=0x7fbdbc06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdbb7fe700 1 -- 172.21.15.22:0/1314441450 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc06d030 con 0x7fbdbc06a710 2023-07-06T01:21:53.205 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdc0872700 1 --2- 172.21.15.22:0/1314441450 >> v2:172.21.15.103:3300/0 conn(0x7fbdbc0cbc10 0x7fbdbc0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:53.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.200+0000 7fbdbb7fe700 1 --2- 172.21.15.22:0/1314441450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdbc06a710 0x7fbdbc06caf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0ca740 tx=0x7fbdac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd475135756462b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.201+0000 7fbdbaffd700 1 -- 172.21.15.22:0/1314441450 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00b870 con 0x7fbdbc06a710 2023-07-06T01:21:53.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.201+0000 7fbdbaffd700 1 -- 172.21.15.22:0/1314441450 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac00b9d0 con 0x7fbdbc06a710 2023-07-06T01:21:53.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.201+0000 7fbdbaffd700 1 -- 172.21.15.22:0/1314441450 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00bd60 con 0x7fbdbc06a710 2023-07-06T01:21:53.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.201+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1314441450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdbc06a710 msgr2=0x7fbdbc06caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.206 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.201+0000 7fbdc22d5700 1 --2- 172.21.15.22:0/1314441450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdbc06a710 0x7fbdbc06caf0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0ca740 tx=0x7fbdac009bd0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.201+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1314441450 shutdown_connections 2023-07-06T01:21:53.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.201+0000 7fbdc22d5700 1 --2- 172.21.15.22:0/1314441450 >> v2:172.21.15.103:3300/0 conn(0x7fbdbc0cbc10 0x7fbdbc0cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.201+0000 7fbdc22d5700 1 --2- 172.21.15.22:0/1314441450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdbc06a710 0x7fbdbc06caf0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.201+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1314441450 >> 172.21.15.22:0/1314441450 conn(0x7fbdbc0b1890 msgr2=0x7fbdbc0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:53.207 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.201+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1314441450 shutdown_connections 2023-07-06T01:21:53.208 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.201+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1314441450 wait complete. 2023-07-06T01:21:53.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.202+0000 7fbdc22d5700 1 Processor -- start 2023-07-06T01:21:53.210 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.203+0000 7fbdc22d5700 1 -- start start 2023-07-06T01:21:53.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.203+0000 7fbdc22d5700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbdbc06a710 0x7fbdbc1f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.203+0000 7fbdc22d5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdbc0af2a0 0x7fbdbc0b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.203+0000 7fbdc22d5700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdbc0cbc10 0x7fbdbc0b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.203+0000 7fbdc22d5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc15f640 con 0x7fbdbc0af2a0 2023-07-06T01:21:53.211 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.203+0000 7fbdc22d5700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc15cdf0 con 0x7fbdbc0cbc10 2023-07-06T01:21:53.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.203+0000 7fbdc22d5700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc15d0d0 con 0x7fbdbc06a710 2023-07-06T01:21:53.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.203+0000 7fbdbb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdbc0af2a0 0x7fbdbc0b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.203+0000 7fbdbb7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdbc0af2a0 0x7fbdbc0b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48296/0 (socket says 172.21.15.22:48296) 2023-07-06T01:21:53.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.203+0000 7fbdbb7fe700 1 -- 172.21.15.22:0/1718254764 learned_addr learned my addr 172.21.15.22:0/1718254764 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:53.212 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.203+0000 7fbdc0872700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdbc0cbc10 0x7fbdbc0b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdbbfff700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbdbc06a710 0x7fbdbc1f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdbb7fe700 1 -- 172.21.15.22:0/1718254764 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbdbc06a710 msgr2=0x7fbdbc1f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdbb7fe700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbdbc06a710 0x7fbdbc1f3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdbb7fe700 1 -- 172.21.15.22:0/1718254764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdbc0cbc10 msgr2=0x7fbdbc0b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.213 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdbb7fe700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdbc0cbc10 0x7fbdbc0b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdbb7fe700 1 -- 172.21.15.22:0/1718254764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdac009550 con 0x7fbdbc0af2a0 2023-07-06T01:21:53.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdbbfff700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbdbc06a710 0x7fbdbc1f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:53.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdc0872700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbdbc0cbc10 0x7fbdbc0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:53.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdbb7fe700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdbc0af2a0 0x7fbdbc0b4640 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbdac000c00 tx=0x7fbdac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.214 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdb8ff9700 1 -- 172.21.15.22:0/1718254764 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac004540 con 0x7fbdbc0af2a0 2023-07-06T01:21:53.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdb8ff9700 1 -- 172.21.15.22:0/1718254764 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac0036a0 con 0x7fbdbc0af2a0 2023-07-06T01:21:53.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1718254764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdbc15d3b0 con 0x7fbdbc0af2a0 2023-07-06T01:21:53.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdb8ff9700 1 -- 172.21.15.22:0/1718254764 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac003a30 con 0x7fbdbc0af2a0 2023-07-06T01:21:53.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.204+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1718254764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdbc15d970 con 0x7fbdbc0af2a0 2023-07-06T01:21:53.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.205+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1718254764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbdbc0cccc0 con 0x7fbdbc0af2a0 2023-07-06T01:21:53.215 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.205+0000 7fbdb8ff9700 1 -- 172.21.15.22:0/1718254764 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbdac021070 con 0x7fbdbc0af2a0 2023-07-06T01:21:53.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.206+0000 7fbdb8ff9700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbd9c06ace0 0x7fbd9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.206+0000 7fbdb8ff9700 1 -- 172.21.15.22:0/1718254764 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbdac08aae0 con 0x7fbdbc0af2a0 2023-07-06T01:21:53.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.206+0000 7fbdb8ff9700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] conn(0x7fbd9c071150 0x7fbd9c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.216 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.206+0000 7fbdb8ff9700 1 -- 172.21.15.22:0/1718254764 --> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd9c073c30 con 0x7fbd9c071150 2023-07-06T01:21:53.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.206+0000 7fbdb8ff9700 1 -- 172.21.15.22:0/1718254764 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdac08af10 con 0x7fbdbc0af2a0 2023-07-06T01:21:53.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.206+0000 7fbdbbfff700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbd9c06ace0 0x7fbd9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.206+0000 7fbdc0872700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] conn(0x7fbd9c071150 0x7fbd9c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.206+0000 7fbdc0872700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] conn(0x7fbd9c071150 0x7fbd9c073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.207+0000 7fbdbbfff700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbd9c06ace0 0x7fbd9c06d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbda4000f60 tx=0x7fbda4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.217 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.207+0000 7fbdb8ff9700 1 -- 172.21.15.22:0/1718254764 <== osd.3 v2:172.21.15.22:6824/3305852500 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbd9c073c30 con 0x7fbd9c071150 2023-07-06T01:21:53.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.221+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1718254764 --> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbdbc0cc9e0 con 0x7fbd9c071150 2023-07-06T01:21:53.223 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.222+0000 7fbdb8ff9700 1 -- 172.21.15.22:0/1718254764 <== osd.3 v2:172.21.15.22:6824/3305852500 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbdb4009b30 con 0x7fbd9c071150 2023-07-06T01:21:53.224 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:21:53.224 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T01:21:53.224 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:21:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.222+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1718254764 >> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] conn(0x7fbd9c071150 msgr2=0x7fbd9c073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.225 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.222+0000 7fbdc22d5700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.22:6824/3305852500,v1:172.21.15.22:6825/3305852500] conn(0x7fbd9c071150 0x7fbd9c073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.222+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1718254764 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbd9c06ace0 msgr2=0x7fbd9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.222+0000 7fbdc22d5700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fbd9c06ace0 0x7fbd9c06d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbda4000f60 tx=0x7fbda4009040 comp rx=0 tx=0).stop 2023-07-06T01:21:53.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.222+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1718254764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdbc0af2a0 msgr2=0x7fbdbc0b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.222+0000 7fbdc22d5700 1 --2- 172.21.15.22:0/1718254764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdbc0af2a0 0x7fbdbc0b4640 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbdac000c00 tx=0x7fbdac004750 comp rx=0 tx=0).stop 2023-07-06T01:21:53.226 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.222+0000 7fbdc0872700 1 -- 172.21.15.22:0/1718254764 reap_dead start 2023-07-06T01:21:53.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.222+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1718254764 shutdown_connections 2023-07-06T01:21:53.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.222+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1718254764 >> 172.21.15.22:0/1718254764 conn(0x7fbdbc0b1890 msgr2=0x7fbdbc15bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:53.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.222+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1718254764 shutdown_connections 2023-07-06T01:21:53.227 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.222+0000 7fbdc22d5700 1 -- 172.21.15.22:0/1718254764 wait complete. 2023-07-06T01:21:53.235 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-07-06T01:21:53.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.360+0000 7fb2bae71700 1 Processor -- start 2023-07-06T01:21:53.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.360+0000 7fb2bae71700 1 -- start start 2023-07-06T01:21:53.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.361+0000 7fb2bae71700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2b40af2a0 0x7fb2b406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.364 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.361+0000 7fb2bae71700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb2b406a000 0x7fb2b406a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.361+0000 7fb2bae71700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b40af670 con 0x7fb2b40cbc10 2023-07-06T01:21:53.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.361+0000 7fb2bae71700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b406a9e0 con 0x7fb2b406a000 2023-07-06T01:21:53.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.361+0000 7fb2bae71700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b40cc7a0 con 0x7fb2b40af2a0 2023-07-06T01:21:53.365 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.361+0000 7fb2b3fff700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb2b406a000 0x7fb2b406a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.361+0000 7fb2b940e700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb2b40cbc10 0x7fb2b40cc090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:39050/0 (socket says 172.21.15.22:39050) 2023-07-06T01:21:53.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.361+0000 7fb2b940e700 1 -- 172.21.15.22:0/735383830 learned_addr learned my addr 172.21.15.22:0/735383830 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:53.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.361+0000 7fb2b8c0d700 1 --2- 172.21.15.22:0/735383830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2b40af2a0 0x7fb2b406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.362+0000 7fb2b3fff700 1 -- 172.21.15.22:0/735383830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2b40af2a0 msgr2=0x7fb2b406caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:53.366 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.362+0000 7fb2b3fff700 1 --2- 172.21.15.22:0/735383830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2b40af2a0 0x7fb2b406caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.362+0000 7fb2b3fff700 1 -- 172.21.15.22:0/735383830 >> v1:172.21.15.22:6789/0 conn(0x7fb2b40cbc10 legacy=0x7fb2b40cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.362+0000 7fb2b3fff700 1 -- 172.21.15.22:0/735383830 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b406d030 con 0x7fb2b406a000 2023-07-06T01:21:53.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.362+0000 7fb2b3fff700 1 --2- 172.21.15.22:0/735383830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2b406a000 0x7fb2b406a4a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009840 tx=0x7fb2a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1219b22a61999401 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.362+0000 7fb2b37fe700 1 -- 172.21.15.22:0/735383830 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0011830 con 0x7fb2b406a000 2023-07-06T01:21:53.367 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.362+0000 7fb2b37fe700 1 -- 172.21.15.22:0/735383830 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a0011990 con 0x7fb2b406a000 2023-07-06T01:21:53.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.362+0000 7fb2b37fe700 1 -- 172.21.15.22:0/735383830 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0011d20 con 0x7fb2b406a000 2023-07-06T01:21:53.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.362+0000 7fb2bae71700 1 -- 172.21.15.22:0/735383830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2b406a000 msgr2=0x7fb2b406a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.362+0000 7fb2bae71700 1 --2- 172.21.15.22:0/735383830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2b406a000 0x7fb2b406a4a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009840 tx=0x7fb2a0009b50 comp rx=0 tx=0).stop 2023-07-06T01:21:53.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.363+0000 7fb2bae71700 1 -- 172.21.15.22:0/735383830 shutdown_connections 2023-07-06T01:21:53.368 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.363+0000 7fb2bae71700 1 --2- 172.21.15.22:0/735383830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2b40af2a0 0x7fb2b406caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.363+0000 7fb2bae71700 1 --2- 172.21.15.22:0/735383830 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2b406a000 0x7fb2b406a4a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.363+0000 7fb2bae71700 1 -- 172.21.15.22:0/735383830 >> 172.21.15.22:0/735383830 conn(0x7fb2b40b1890 msgr2=0x7fb2b40b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:53.369 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.363+0000 7fb2bae71700 1 -- 172.21.15.22:0/735383830 shutdown_connections 2023-07-06T01:21:53.370 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.363+0000 7fb2bae71700 1 -- 172.21.15.22:0/735383830 wait complete. 2023-07-06T01:21:53.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.364+0000 7fb2bae71700 1 Processor -- start 2023-07-06T01:21:53.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.364+0000 7fb2bae71700 1 -- start start 2023-07-06T01:21:53.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2bae71700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2b406a000 0x7fb2b4158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2bae71700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2b40af2a0 0x7fb2b40c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.373 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2bae71700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2b40cbc10 0x7fb2b406e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2bae71700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b4158e00 con 0x7fb2b40af2a0 2023-07-06T01:21:53.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2bae71700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b415ad10 con 0x7fb2b406a000 2023-07-06T01:21:53.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2bae71700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b4158890 con 0x7fb2b40cbc10 2023-07-06T01:21:53.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2b3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2b40af2a0 0x7fb2b40c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.374 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2b3fff700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2b40af2a0 0x7fb2b40c33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48310/0 (socket says 172.21.15.22:48310) 2023-07-06T01:21:53.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2b3fff700 1 -- 172.21.15.22:0/2203620879 learned_addr learned my addr 172.21.15.22:0/2203620879 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:53.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2b8c0d700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2b406a000 0x7fb2b4158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2b940e700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2b40cbc10 0x7fb2b406e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2b3fff700 1 -- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2b40cbc10 msgr2=0x7fb2b406e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.375 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2b3fff700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2b40cbc10 0x7fb2b406e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2b3fff700 1 -- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2b406a000 msgr2=0x7fb2b4158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2b3fff700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2b406a000 0x7fb2b4158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2b3fff700 1 -- 172.21.15.22:0/2203620879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00094a0 con 0x7fb2b40af2a0 2023-07-06T01:21:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2b3fff700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2b40af2a0 0x7fb2b40c33f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0000c00 tx=0x7fb2a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.376 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.365+0000 7fb2b17fa700 1 -- 172.21.15.22:0/2203620879 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a00036a0 con 0x7fb2b40af2a0 2023-07-06T01:21:53.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.366+0000 7fb2b17fa700 1 -- 172.21.15.22:0/2203620879 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a001f070 con 0x7fb2b40af2a0 2023-07-06T01:21:53.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.366+0000 7fb2bae71700 1 -- 172.21.15.22:0/2203620879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b41f6550 con 0x7fb2b40af2a0 2023-07-06T01:21:53.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.366+0000 7fb2b17fa700 1 -- 172.21.15.22:0/2203620879 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0003a60 con 0x7fb2b40af2a0 2023-07-06T01:21:53.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.366+0000 7fb2bae71700 1 -- 172.21.15.22:0/2203620879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b4158a90 con 0x7fb2b40af2a0 2023-07-06T01:21:53.377 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.366+0000 7fb2bae71700 1 -- 172.21.15.22:0/2203620879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b415ca90 con 0x7fb2b40af2a0 2023-07-06T01:21:53.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.367+0000 7fb2b17fa700 1 -- 172.21.15.22:0/2203620879 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb2a0003bc0 con 0x7fb2b40af2a0 2023-07-06T01:21:53.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.367+0000 7fb2b17fa700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb29806ace0 0x7fb29806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.367+0000 7fb2b17fa700 1 -- 172.21.15.22:0/2203620879 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb2a0021070 con 0x7fb2b40af2a0 2023-07-06T01:21:53.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.367+0000 7fb2b8c0d700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb29806ace0 0x7fb29806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.378 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.368+0000 7fb2b8c0d700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb29806ace0 0x7fb29806d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009be0 tx=0x7fb2a8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.379 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.372+0000 7fb2b17fa700 1 -- 172.21.15.22:0/2203620879 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb2b415ca90 con 0x7fb2b40af2a0 2023-07-06T01:21:53.570 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.569+0000 7fb2bae71700 1 -- 172.21.15.22:0/2203620879 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2b40cd560 con 0x7fb29806ace0 2023-07-06T01:21:53.571 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.569+0000 7fb2b17fa700 1 -- 172.21.15.22:0/2203620879 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb2a8004020 con 0x7fb29806ace0 2023-07-06T01:21:53.573 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.572+0000 7fb2bae71700 1 -- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb29806ace0 msgr2=0x7fb29806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.572+0000 7fb2bae71700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb29806ace0 0x7fb29806d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009be0 tx=0x7fb2a8009210 comp rx=0 tx=0).stop 2023-07-06T01:21:53.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.572+0000 7fb2bae71700 1 -- 172.21.15.22:0/2203620879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2b40af2a0 msgr2=0x7fb2b40c33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.572+0000 7fb2bae71700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2b40af2a0 0x7fb2b40c33f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0000c00 tx=0x7fb2a0004750 comp rx=0 tx=0).stop 2023-07-06T01:21:53.574 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.572+0000 7fb2bae71700 1 -- 172.21.15.22:0/2203620879 shutdown_connections 2023-07-06T01:21:53.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.572+0000 7fb2bae71700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2b40cbc10 0x7fb2b406e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.572+0000 7fb2bae71700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fb29806ace0 0x7fb29806d190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.572+0000 7fb2bae71700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2b406a000 0x7fb2b4158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.572+0000 7fb2bae71700 1 --2- 172.21.15.22:0/2203620879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2b40af2a0 0x7fb2b40c33f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.575 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.572+0000 7fb2bae71700 1 -- 172.21.15.22:0/2203620879 >> 172.21.15.22:0/2203620879 conn(0x7fb2b40b1890 msgr2=0x7fb2b40cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.572+0000 7fb2bae71700 1 -- 172.21.15.22:0/2203620879 shutdown_connections 2023-07-06T01:21:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.572+0000 7fb2bae71700 1 -- 172.21.15.22:0/2203620879 wait complete. 2023-07-06T01:21:53.576 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 3 to deep-scrub 2023-07-06T01:21:53.586 INFO:tasks.ceph:Scrubbing osd.4 2023-07-06T01:21:53.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-07-06T01:21:53.715 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.714+0000 7f5afe6ea700 1 Processor -- start 2023-07-06T01:21:53.716 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.714+0000 7f5afe6ea700 1 -- start start 2023-07-06T01:21:53.718 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5afe6ea700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f5af815a710 0x7f5af80ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5afe6ea700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5af80aecb0 0x7f5af80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5afe6ea700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af80af270 con 0x7f5af80ceda0 2023-07-06T01:21:53.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5afe6ea700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af80cf220 con 0x7f5af815a710 2023-07-06T01:21:53.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5afe6ea700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80af550 con 0x7f5af80aecb0 2023-07-06T01:21:53.719 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5afcc87700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f5af80ceda0 0x7f5af815e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:39064/0 (socket says 172.21.15.22:39064) 2023-07-06T01:21:53.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5afcc87700 1 -- 172.21.15.22:0/2063474483 learned_addr learned my addr 172.21.15.22:0/2063474483 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:53.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5af77fe700 1 --2- 172.21.15.22:0/2063474483 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5af80aecb0 0x7f5af80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5af7fff700 1 --2- 172.21.15.22:0/2063474483 >> v2:172.21.15.103:3300/0 conn(0x7f5af815a710 0x7f5af80ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5af6ffd700 1 -- 172.21.15.22:0/2063474483 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423564468 0 0) 0x7f5af80af270 con 0x7f5af80ceda0 2023-07-06T01:21:53.720 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5af6ffd700 1 -- 172.21.15.22:0/2063474483 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ae40035b0 con 0x7f5af80ceda0 2023-07-06T01:21:53.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5af77fe700 1 -- 172.21.15.22:0/2063474483 >> v2:172.21.15.103:3300/0 conn(0x7f5af815a710 msgr2=0x7f5af80ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:53.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5af77fe700 1 --2- 172.21.15.22:0/2063474483 >> v2:172.21.15.103:3300/0 conn(0x7f5af815a710 0x7f5af80ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5af77fe700 1 -- 172.21.15.22:0/2063474483 >> v1:172.21.15.22:6789/0 conn(0x7f5af80ceda0 legacy=0x7f5af815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5af77fe700 1 -- 172.21.15.22:0/2063474483 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af80af830 con 0x7f5af80aecb0 2023-07-06T01:21:53.721 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.715+0000 7f5af7fff700 1 --2- 172.21.15.22:0/2063474483 >> v2:172.21.15.103:3300/0 conn(0x7f5af815a710 0x7f5af80ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.716+0000 7f5af77fe700 1 --2- 172.21.15.22:0/2063474483 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5af80aecb0 0x7f5af80ce860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009840 tx=0x7f5ae8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71c80ee3c374b6dd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.716+0000 7f5af6ffd700 1 -- 172.21.15.22:0/2063474483 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8011830 con 0x7f5af80aecb0 2023-07-06T01:21:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.716+0000 7f5af6ffd700 1 -- 172.21.15.22:0/2063474483 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae8011990 con 0x7f5af80aecb0 2023-07-06T01:21:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.716+0000 7f5af6ffd700 1 -- 172.21.15.22:0/2063474483 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8011d20 con 0x7f5af80aecb0 2023-07-06T01:21:53.722 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.716+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2063474483 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5af80aecb0 msgr2=0x7f5af80ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.716+0000 7f5afe6ea700 1 --2- 172.21.15.22:0/2063474483 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5af80aecb0 0x7f5af80ce860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009840 tx=0x7f5ae8009b50 comp rx=0 tx=0).stop 2023-07-06T01:21:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.716+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2063474483 shutdown_connections 2023-07-06T01:21:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.716+0000 7f5afe6ea700 1 --2- 172.21.15.22:0/2063474483 >> v2:172.21.15.103:3300/0 conn(0x7f5af815a710 0x7f5af80ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.716+0000 7f5afe6ea700 1 --2- 172.21.15.22:0/2063474483 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5af80aecb0 0x7f5af80ce860 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.716+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2063474483 >> 172.21.15.22:0/2063474483 conn(0x7f5af80999f0 msgr2=0x7f5af8099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:53.723 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.716+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2063474483 shutdown_connections 2023-07-06T01:21:53.724 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.717+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2063474483 wait complete. 2023-07-06T01:21:53.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.718+0000 7f5afe6ea700 1 Processor -- start 2023-07-06T01:21:53.726 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.718+0000 7f5afe6ea700 1 -- start start 2023-07-06T01:21:53.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afe6ea700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5af80aecb0 0x7f5af8153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afe6ea700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5af80ceda0 0x7f5af80b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afe6ea700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5af815a710 0x7f5af80b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.727 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afe6ea700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af8153ab0 con 0x7f5af815a710 2023-07-06T01:21:53.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afe6ea700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80ce1c0 con 0x7f5af80aecb0 2023-07-06T01:21:53.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afe6ea700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80c4160 con 0x7f5af80ceda0 2023-07-06T01:21:53.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afcc87700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5af815a710 0x7f5af80b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afcc87700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5af815a710 0x7f5af80b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48326/0 (socket says 172.21.15.22:48326) 2023-07-06T01:21:53.728 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afcc87700 1 -- 172.21.15.22:0/2795226245 learned_addr learned my addr 172.21.15.22:0/2795226245 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5af77fe700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5af80ceda0 0x7f5af80b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5af7fff700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5af80aecb0 0x7f5af8153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afcc87700 1 -- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5af80ceda0 msgr2=0x7f5af80b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afcc87700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5af80ceda0 0x7f5af80b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.729 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afcc87700 1 -- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5af80aecb0 msgr2=0x7f5af8153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afcc87700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5af80aecb0 0x7f5af8153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afcc87700 1 -- 172.21.15.22:0/2795226245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae00094a0 con 0x7f5af815a710 2023-07-06T01:21:53.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afcc87700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5af815a710 0x7f5af80b4090 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5af000cb40 tx=0x7f5af000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5af4ff9700 1 -- 172.21.15.22:0/2795226245 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af000a9e0 con 0x7f5af815a710 2023-07-06T01:21:53.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5af4ff9700 1 -- 172.21.15.22:0/2795226245 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af000ab40 con 0x7f5af815a710 2023-07-06T01:21:53.730 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2795226245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae80094a0 con 0x7f5af815a710 2023-07-06T01:21:53.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5af4ff9700 1 -- 172.21.15.22:0/2795226245 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0012400 con 0x7f5af815a710 2023-07-06T01:21:53.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.719+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2795226245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af80c61f0 con 0x7f5af815a710 2023-07-06T01:21:53.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.720+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2795226245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5af80c6070 con 0x7f5af815a710 2023-07-06T01:21:53.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.721+0000 7f5af4ff9700 1 -- 172.21.15.22:0/2795226245 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5af001b3b0 con 0x7f5af815a710 2023-07-06T01:21:53.731 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.721+0000 7f5af4ff9700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5ad806ace0 0x7f5ad806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.721+0000 7f5af4ff9700 1 -- 172.21.15.22:0/2795226245 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5af0088cd0 con 0x7f5af815a710 2023-07-06T01:21:53.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.721+0000 7f5af4ff9700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] conn(0x7f5ad8071150 0x7f5ad8073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.721+0000 7f5af7fff700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5ad806ace0 0x7f5ad806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.721+0000 7f5af4ff9700 1 -- 172.21.15.22:0/2795226245 --> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ad8073c30 con 0x7f5ad8071150 2023-07-06T01:21:53.732 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.722+0000 7f5af4ff9700 1 -- 172.21.15.22:0/2795226245 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5af008b370 con 0x7f5af815a710 2023-07-06T01:21:53.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.722+0000 7f5af77fe700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] conn(0x7f5ad8071150 0x7f5ad8073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.722+0000 7f5af7fff700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5ad806ace0 0x7f5ad806d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009d70 tx=0x7f5ae0009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.722+0000 7f5af77fe700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] conn(0x7f5ad8071150 0x7f5ad8073560 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.733 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.723+0000 7f5af4ff9700 1 -- 172.21.15.22:0/2795226245 <== osd.4 v2:172.21.15.103:6824/3712184942 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5ad8073c30 con 0x7f5ad8071150 2023-07-06T01:21:53.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.737+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2795226245 --> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5af80cb5c0 con 0x7f5ad8071150 2023-07-06T01:21:53.739 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.738+0000 7f5af4ff9700 1 -- 172.21.15.22:0/2795226245 <== osd.4 v2:172.21.15.103:6824/3712184942 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5ae8011630 con 0x7f5ad8071150 2023-07-06T01:21:53.740 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:21:53.740 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T01:21:53.740 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:21:53.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.738+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] conn(0x7f5ad8071150 msgr2=0x7f5ad8073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.741 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.738+0000 7f5afe6ea700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:6824/3712184942,v1:172.21.15.103:6825/3712184942] conn(0x7f5ad8071150 0x7f5ad8073560 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.738+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5ad806ace0 msgr2=0x7f5ad806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.738+0000 7f5afe6ea700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f5ad806ace0 0x7f5ad806d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009d70 tx=0x7f5ae0009210 comp rx=0 tx=0).stop 2023-07-06T01:21:53.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.738+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2795226245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5af815a710 msgr2=0x7f5af80b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.738+0000 7f5afe6ea700 1 --2- 172.21.15.22:0/2795226245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5af815a710 0x7f5af80b4090 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5af000cb40 tx=0x7f5af000ce50 comp rx=0 tx=0).stop 2023-07-06T01:21:53.742 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.738+0000 7f5afcc87700 1 -- 172.21.15.22:0/2795226245 reap_dead start 2023-07-06T01:21:53.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.738+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2795226245 shutdown_connections 2023-07-06T01:21:53.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.738+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2795226245 >> 172.21.15.22:0/2795226245 conn(0x7f5af80999f0 msgr2=0x7f5af81583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:53.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.738+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2795226245 shutdown_connections 2023-07-06T01:21:53.743 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.738+0000 7f5afe6ea700 1 -- 172.21.15.22:0/2795226245 wait complete. 2023-07-06T01:21:53.752 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-07-06T01:21:53.877 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.876+0000 7f1d115f6700 1 Processor -- start 2023-07-06T01:21:53.878 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.876+0000 7f1d115f6700 1 -- start start 2023-07-06T01:21:53.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d115f6700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f1d0c0b3b70 0x7f1d0c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d115f6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d0c0b40c0 0x7f1d0c0d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.880 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d115f6700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0c0cebd0 con 0x7f1d0c0b36f0 2023-07-06T01:21:53.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d115f6700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d0c15cff0 con 0x7f1d0c0b3b70 2023-07-06T01:21:53.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d115f6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0cdfc0 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d0affd700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1d0c0b36f0 0x7f1d0c0cf060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:39076/0 (socket says 172.21.15.22:39076) 2023-07-06T01:21:53.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d0affd700 1 -- 172.21.15.22:0/3148127087 learned_addr learned my addr 172.21.15.22:0/3148127087 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:53.881 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d0b7fe700 1 --2- 172.21.15.22:0/3148127087 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d0c0b40c0 0x7f1d0c0d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d0a7fc700 1 --2- 172.21.15.22:0/3148127087 >> v2:172.21.15.103:3300/0 conn(0x7f1d0c0b3b70 0x7f1d0c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d09ffb700 1 -- 172.21.15.22:0/3148127087 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026774509 0 0) 0x7f1d0c0cebd0 con 0x7f1d0c0b36f0 2023-07-06T01:21:53.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d09ffb700 1 -- 172.21.15.22:0/3148127087 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf40035b0 con 0x7f1d0c0b36f0 2023-07-06T01:21:53.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d0b7fe700 1 -- 172.21.15.22:0/3148127087 >> v2:172.21.15.103:3300/0 conn(0x7f1d0c0b3b70 msgr2=0x7f1d0c0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:53.882 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d0b7fe700 1 --2- 172.21.15.22:0/3148127087 >> v2:172.21.15.103:3300/0 conn(0x7f1d0c0b3b70 0x7f1d0c0d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d0b7fe700 1 -- 172.21.15.22:0/3148127087 >> v1:172.21.15.22:6789/0 conn(0x7f1d0c0b36f0 legacy=0x7f1d0c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d0b7fe700 1 -- 172.21.15.22:0/3148127087 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c0ce2a0 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.877+0000 7f1d0b7fe700 1 --2- 172.21.15.22:0/3148127087 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d0c0b40c0 0x7f1d0c0d0fd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009d90 tx=0x7f1d0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c29ff53e9cbf37b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.878+0000 7f1d09ffb700 1 -- 172.21.15.22:0/3148127087 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0400e070 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.883 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.878+0000 7f1d09ffb700 1 -- 172.21.15.22:0/3148127087 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0400bb20 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.878+0000 7f1d09ffb700 1 -- 172.21.15.22:0/3148127087 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04012430 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.878+0000 7f1d115f6700 1 -- 172.21.15.22:0/3148127087 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d0c0b40c0 msgr2=0x7f1d0c0d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.878+0000 7f1d115f6700 1 --2- 172.21.15.22:0/3148127087 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d0c0b40c0 0x7f1d0c0d0fd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009d90 tx=0x7f1d0400b330 comp rx=0 tx=0).stop 2023-07-06T01:21:53.884 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.878+0000 7f1d115f6700 1 -- 172.21.15.22:0/3148127087 shutdown_connections 2023-07-06T01:21:53.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.878+0000 7f1d115f6700 1 --2- 172.21.15.22:0/3148127087 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d0c0b40c0 0x7f1d0c0d0fd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.878+0000 7f1d115f6700 1 --2- 172.21.15.22:0/3148127087 >> v2:172.21.15.103:3300/0 conn(0x7f1d0c0b3b70 0x7f1d0c0d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.878+0000 7f1d115f6700 1 -- 172.21.15.22:0/3148127087 >> 172.21.15.22:0/3148127087 conn(0x7f1d0c09c2e0 msgr2=0x7f1d0c0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:53.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.878+0000 7f1d115f6700 1 -- 172.21.15.22:0/3148127087 shutdown_connections 2023-07-06T01:21:53.885 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.878+0000 7f1d115f6700 1 -- 172.21.15.22:0/3148127087 wait complete. 2023-07-06T01:21:53.889 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.880+0000 7f1d115f6700 1 Processor -- start 2023-07-06T01:21:53.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.880+0000 7f1d115f6700 1 -- start start 2023-07-06T01:21:53.890 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.880+0000 7f1d115f6700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d0c0b36f0 0x7f1d0c15a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.880+0000 7f1d115f6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d0c0b3b70 0x7f1d0c0b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.891 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.880+0000 7f1d115f6700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c0b40c0 0x7f1d0c15e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.880+0000 7f1d115f6700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0b60f0 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.880+0000 7f1d115f6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c15de70 con 0x7f1d0c0b3b70 2023-07-06T01:21:53.892 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.880+0000 7f1d115f6700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c15e150 con 0x7f1d0c0b36f0 2023-07-06T01:21:53.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1d0b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c0b40c0 0x7f1d0c15e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1d0b7fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c0b40c0 0x7f1d0c15e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48342/0 (socket says 172.21.15.22:48342) 2023-07-06T01:21:53.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1d0b7fe700 1 -- 172.21.15.22:0/1801102748 learned_addr learned my addr 172.21.15.22:0/1801102748 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:53.893 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1d0a7fc700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d0c0b3b70 0x7f1d0c0b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1d0affd700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d0c0b36f0 0x7f1d0c15a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1d0b7fe700 1 -- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d0c0b36f0 msgr2=0x7f1d0c15a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1d0b7fe700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d0c0b36f0 0x7f1d0c15a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.894 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1d0b7fe700 1 -- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d0c0b3b70 msgr2=0x7f1d0c0b7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1d0b7fe700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d0c0b3b70 0x7f1d0c0b7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1d0b7fe700 1 -- 172.21.15.22:0/1801102748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04009a70 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1d0b7fe700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c0b40c0 0x7f1d0c15e810 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1d04014040 tx=0x7f1d04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1cf37fe700 1 -- 172.21.15.22:0/1801102748 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04012a90 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.895 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1cf37fe700 1 -- 172.21.15.22:0/1801102748 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d04020070 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1d115f6700 1 -- 172.21.15.22:0/1801102748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0c15e430 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1cf37fe700 1 -- 172.21.15.22:0/1801102748 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04012e50 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.881+0000 7f1d115f6700 1 -- 172.21.15.22:0/1801102748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0c0ca4c0 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.896 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.882+0000 7f1d115f6700 1 -- 172.21.15.22:0/1801102748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0c0b4840 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.883+0000 7f1cf37fe700 1 -- 172.21.15.22:0/1801102748 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d04004040 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.883+0000 7f1cf37fe700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1cf806ace0 0x7f1cf806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:53.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.883+0000 7f1cf37fe700 1 -- 172.21.15.22:0/1801102748 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1d04022070 con 0x7f1d0c0b40c0 2023-07-06T01:21:53.897 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.883+0000 7f1d0affd700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1cf806ace0 0x7f1cf806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:53.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.884+0000 7f1d0affd700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1cf806ace0 0x7f1cf806d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc007fd0 tx=0x7f1cfc007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:53.898 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:53.887+0000 7f1cf37fe700 1 -- 172.21.15.22:0/1801102748 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1d04056530 con 0x7f1d0c0b40c0 2023-07-06T01:21:54.087 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.085+0000 7f1d115f6700 1 -- 172.21.15.22:0/1801102748 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d0c0ce860 con 0x7f1cf806ace0 2023-07-06T01:21:54.088 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.086+0000 7f1cf37fe700 1 -- 172.21.15.22:0/1801102748 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1cfc0058e0 con 0x7f1cf806ace0 2023-07-06T01:21:54.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.088+0000 7f1d115f6700 1 -- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1cf806ace0 msgr2=0x7f1cf806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.088+0000 7f1d115f6700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1cf806ace0 0x7f1cf806d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc007fd0 tx=0x7f1cfc007dc0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.090 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.088+0000 7f1d115f6700 1 -- 172.21.15.22:0/1801102748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c0b40c0 msgr2=0x7f1d0c15e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.088+0000 7f1d115f6700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c0b40c0 0x7f1d0c15e810 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1d04014040 tx=0x7f1d04004ad0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.088+0000 7f1d115f6700 1 -- 172.21.15.22:0/1801102748 shutdown_connections 2023-07-06T01:21:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.088+0000 7f1d115f6700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d0c0b40c0 0x7f1d0c15e810 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.088+0000 7f1d115f6700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d0c0b36f0 0x7f1d0c15a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.091 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.088+0000 7f1d115f6700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1cf806ace0 0x7f1cf806d190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.088+0000 7f1d115f6700 1 --2- 172.21.15.22:0/1801102748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d0c0b3b70 0x7f1d0c0b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.088+0000 7f1d115f6700 1 -- 172.21.15.22:0/1801102748 >> 172.21.15.22:0/1801102748 conn(0x7f1d0c09c2e0 msgr2=0x7f1d0c06bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:54.092 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.088+0000 7f1d115f6700 1 -- 172.21.15.22:0/1801102748 shutdown_connections 2023-07-06T01:21:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.088+0000 7f1d115f6700 1 -- 172.21.15.22:0/1801102748 wait complete. 2023-07-06T01:21:54.093 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 4 to deep-scrub 2023-07-06T01:21:54.102 INFO:tasks.ceph:Scrubbing osd.5 2023-07-06T01:21:54.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-07-06T01:21:54.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.232+0000 7fcaee716700 1 Processor -- start 2023-07-06T01:21:54.234 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.232+0000 7fcaee716700 1 -- start start 2023-07-06T01:21:54.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.233+0000 7fcaee716700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcae80cf260 0x7fcae8167a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.233+0000 7fcaee716700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae80ca2b0 con 0x7fcae815cb50 2023-07-06T01:21:54.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.233+0000 7fcaee716700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae80cfc90 con 0x7fcae806a710 2023-07-06T01:21:54.237 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.233+0000 7fcaee716700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae80af440 con 0x7fcae80cf260 2023-07-06T01:21:54.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.233+0000 7fcaeccb3700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fcae80cf260 0x7fcae8167a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.233+0000 7fcae7fff700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fcae815cb50 0x7fcae806a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:39090/0 (socket says 172.21.15.22:39090) 2023-07-06T01:21:54.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.233+0000 7fcae7fff700 1 -- 172.21.15.22:0/570342144 learned_addr learned my addr 172.21.15.22:0/570342144 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:21:54.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.233+0000 7fcaeccb3700 1 -- 172.21.15.22:0/570342144 >> v1:172.21.15.103:6789/0 conn(0x7fcae806a710 legacy=0x7fcae80ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.238 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.233+0000 7fcaeccb3700 1 -- 172.21.15.22:0/570342144 >> v1:172.21.15.22:6789/0 conn(0x7fcae815cb50 legacy=0x7fcae806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.233+0000 7fcae6ffd700 1 -- 172.21.15.22:0/570342144 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453624926 0 0) 0x7fcae80ca2b0 con 0x7fcae815cb50 2023-07-06T01:21:54.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.233+0000 7fcaeccb3700 1 -- 172.21.15.22:0/570342144 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae80cf6a0 con 0x7fcae80cf260 2023-07-06T01:21:54.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.234+0000 7fcaeccb3700 1 --2- 172.21.15.22:0/570342144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae80cf260 0x7fcae8167a40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009d90 tx=0x7fcae000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a18836fd65deca server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.234+0000 7fcae6ffd700 1 -- 172.21.15.22:0/570342144 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000e070 con 0x7fcae80cf260 2023-07-06T01:21:54.239 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.234+0000 7fcae6ffd700 1 -- 172.21.15.22:0/570342144 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae000bbd0 con 0x7fcae80cf260 2023-07-06T01:21:54.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.234+0000 7fcae6ffd700 1 -- 172.21.15.22:0/570342144 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0012430 con 0x7fcae80cf260 2023-07-06T01:21:54.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.234+0000 7fcaee716700 1 -- 172.21.15.22:0/570342144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae80cf260 msgr2=0x7fcae8167a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.234+0000 7fcaee716700 1 --2- 172.21.15.22:0/570342144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae80cf260 0x7fcae8167a40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009d90 tx=0x7fcae000b330 comp rx=0 tx=0).stop 2023-07-06T01:21:54.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.234+0000 7fcaee716700 1 -- 172.21.15.22:0/570342144 shutdown_connections 2023-07-06T01:21:54.240 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.234+0000 7fcaee716700 1 --2- 172.21.15.22:0/570342144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae80cf260 0x7fcae8167a40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.234+0000 7fcaee716700 1 -- 172.21.15.22:0/570342144 >> 172.21.15.22:0/570342144 conn(0x7fcae806b110 msgr2=0x7fcae806b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:54.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.234+0000 7fcaee716700 1 -- 172.21.15.22:0/570342144 shutdown_connections 2023-07-06T01:21:54.241 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.234+0000 7fcaee716700 1 -- 172.21.15.22:0/570342144 wait complete. 2023-07-06T01:21:54.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.236+0000 7fcaee716700 1 Processor -- start 2023-07-06T01:21:54.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.236+0000 7fcaee716700 1 -- start start 2023-07-06T01:21:54.244 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.236+0000 7fcaee716700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcae806a710 0x7fcae806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaee716700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcae80cf260 0x7fcae814bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaee716700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae815cb50 0x7fcae814c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaee716700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80b5c10 con 0x7fcae815cb50 2023-07-06T01:21:54.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaee716700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80c62c0 con 0x7fcae80cf260 2023-07-06T01:21:54.245 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaee716700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80b42e0 con 0x7fcae806a710 2023-07-06T01:21:54.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaeccb3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae815cb50 0x7fcae814c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaeccb3700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae815cb50 0x7fcae814c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48356/0 (socket says 172.21.15.22:48356) 2023-07-06T01:21:54.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaeccb3700 1 -- 172.21.15.22:0/2667196559 learned_addr learned my addr 172.21.15.22:0/2667196559 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:54.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcae7fff700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcae806a710 0x7fcae806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.246 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcae77fe700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcae80cf260 0x7fcae814bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaeccb3700 1 -- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcae806a710 msgr2=0x7fcae806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaeccb3700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcae806a710 0x7fcae806cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaeccb3700 1 -- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcae80cf260 msgr2=0x7fcae814bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaeccb3700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcae80cf260 0x7fcae814bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.247 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaeccb3700 1 -- 172.21.15.22:0/2667196559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae0009a70 con 0x7fcae815cb50 2023-07-06T01:21:54.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcae7fff700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcae806a710 0x7fcae806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:54.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaeccb3700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae815cb50 0x7fcae814c1d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcae0014040 tx=0x7fcae0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcae4ff9700 1 -- 172.21.15.22:0/2667196559 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0012430 con 0x7fcae815cb50 2023-07-06T01:21:54.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcae4ff9700 1 -- 172.21.15.22:0/2667196559 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae0020070 con 0x7fcae815cb50 2023-07-06T01:21:54.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaee716700 1 -- 172.21.15.22:0/2667196559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae80c5990 con 0x7fcae815cb50 2023-07-06T01:21:54.248 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcae4ff9700 1 -- 172.21.15.22:0/2667196559 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0007000 con 0x7fcae815cb50 2023-07-06T01:21:54.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.237+0000 7fcaee716700 1 -- 172.21.15.22:0/2667196559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae80c5f50 con 0x7fcae815cb50 2023-07-06T01:21:54.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.238+0000 7fcaee716700 1 -- 172.21.15.22:0/2667196559 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcae80d0210 con 0x7fcae815cb50 2023-07-06T01:21:54.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.239+0000 7fcae4ff9700 1 -- 172.21.15.22:0/2667196559 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcae0004280 con 0x7fcae815cb50 2023-07-06T01:21:54.249 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.239+0000 7fcae4ff9700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fcac806ace0 0x7fcac806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.239+0000 7fcae4ff9700 1 -- 172.21.15.22:0/2667196559 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fcae0022070 con 0x7fcae815cb50 2023-07-06T01:21:54.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.239+0000 7fcae7fff700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fcac806ace0 0x7fcac806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.239+0000 7fcae4ff9700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] conn(0x7fcac8071150 0x7fcac8073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.239+0000 7fcae4ff9700 1 -- 172.21.15.22:0/2667196559 --> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcac8073c30 con 0x7fcac8071150 2023-07-06T01:21:54.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.239+0000 7fcae4ff9700 1 -- 172.21.15.22:0/2667196559 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcae008da30 con 0x7fcae815cb50 2023-07-06T01:21:54.250 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.240+0000 7fcae77fe700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] conn(0x7fcac8071150 0x7fcac8073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.240+0000 7fcae7fff700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fcac806ace0 0x7fcac806d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcad0000f60 tx=0x7fcad0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.240+0000 7fcae77fe700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] conn(0x7fcac8071150 0x7fcac8073560 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.251 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.241+0000 7fcae4ff9700 1 -- 172.21.15.22:0/2667196559 <== osd.5 v2:172.21.15.103:6800/1893069563 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcac8073c30 con 0x7fcac8071150 2023-07-06T01:21:54.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.255+0000 7fcaee716700 1 -- 172.21.15.22:0/2667196559 --> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcae80b5750 con 0x7fcac8071150 2023-07-06T01:21:54.257 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.255+0000 7fcae4ff9700 1 -- 172.21.15.22:0/2667196559 <== osd.5 v2:172.21.15.103:6800/1893069563 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcae80b5750 con 0x7fcac8071150 2023-07-06T01:21:54.258 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:21:54.259 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T01:21:54.259 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:21:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.256+0000 7fcaee716700 1 -- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] conn(0x7fcac8071150 msgr2=0x7fcac8073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.256+0000 7fcaee716700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:6800/1893069563,v1:172.21.15.103:6801/1893069563] conn(0x7fcac8071150 0x7fcac8073560 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.256+0000 7fcaee716700 1 -- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fcac806ace0 msgr2=0x7fcac806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.262 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.256+0000 7fcaee716700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fcac806ace0 0x7fcac806d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcad0000f60 tx=0x7fcad0009040 comp rx=0 tx=0).stop 2023-07-06T01:21:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.256+0000 7fcaee716700 1 -- 172.21.15.22:0/2667196559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae815cb50 msgr2=0x7fcae814c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.256+0000 7fcaee716700 1 --2- 172.21.15.22:0/2667196559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcae815cb50 0x7fcae814c1d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcae0014040 tx=0x7fcae0004ad0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.256+0000 7fcaeccb3700 1 -- 172.21.15.22:0/2667196559 reap_dead start 2023-07-06T01:21:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.256+0000 7fcaee716700 1 -- 172.21.15.22:0/2667196559 shutdown_connections 2023-07-06T01:21:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.256+0000 7fcaee716700 1 -- 172.21.15.22:0/2667196559 >> 172.21.15.22:0/2667196559 conn(0x7fcae806b110 msgr2=0x7fcae80ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.256+0000 7fcaee716700 1 -- 172.21.15.22:0/2667196559 shutdown_connections 2023-07-06T01:21:54.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.256+0000 7fcaee716700 1 -- 172.21.15.22:0/2667196559 wait complete. 2023-07-06T01:21:54.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-07-06T01:21:54.395 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.394+0000 7f1a3d2af700 1 Processor -- start 2023-07-06T01:21:54.396 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.394+0000 7f1a3d2af700 1 -- start start 2023-07-06T01:21:54.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a3d2af700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1a3806a710 0x7f1a3806caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a3d2af700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a380cbc10 0x7f1a380cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.398 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a3d2af700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a380af670 con 0x7f1a380af2a0 2023-07-06T01:21:54.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a3d2af700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a380cd560 con 0x7f1a3806a710 2023-07-06T01:21:54.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a3d2af700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3815cab0 con 0x7f1a380cbc10 2023-07-06T01:21:54.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a367fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1a3806a710 0x7f1a3806caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.399 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a367fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1a3806a710 0x7f1a3806caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48364/0 (socket says 172.21.15.22:48364) 2023-07-06T01:21:54.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a367fc700 1 -- 172.21.15.22:0/2432558079 learned_addr learned my addr 172.21.15.22:0/2432558079 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:54.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a377fe700 1 --2- 172.21.15.22:0/2432558079 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a380cbc10 0x7f1a380cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a367fc700 1 -- 172.21.15.22:0/2432558079 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a380cbc10 msgr2=0x7f1a380cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:54.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a367fc700 1 --2- 172.21.15.22:0/2432558079 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a380cbc10 0x7f1a380cc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.400 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a367fc700 1 -- 172.21.15.22:0/2432558079 >> v1:172.21.15.103:6789/0 conn(0x7f1a380af2a0 legacy=0x7f1a3806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:54.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a367fc700 1 -- 172.21.15.22:0/2432558079 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3806d030 con 0x7f1a3806a710 2023-07-06T01:21:54.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a367fc700 1 --2- 172.21.15.22:0/2432558079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a3806a710 0x7f1a3806caf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0098f0 tx=0x7f1a2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b135be50b9eef7bb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a35ffb700 1 -- 172.21.15.22:0/2432558079 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c004230 con 0x7f1a3806a710 2023-07-06T01:21:54.401 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a35ffb700 1 -- 172.21.15.22:0/2432558079 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c004390 con 0x7f1a3806a710 2023-07-06T01:21:54.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.395+0000 7f1a35ffb700 1 -- 172.21.15.22:0/2432558079 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0036a0 con 0x7f1a3806a710 2023-07-06T01:21:54.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.396+0000 7f1a3d2af700 1 -- 172.21.15.22:0/2432558079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a3806a710 msgr2=0x7f1a3806caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.396+0000 7f1a3d2af700 1 --2- 172.21.15.22:0/2432558079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a3806a710 0x7f1a3806caf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0098f0 tx=0x7f1a2c009c00 comp rx=0 tx=0).stop 2023-07-06T01:21:54.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.396+0000 7f1a3d2af700 1 -- 172.21.15.22:0/2432558079 shutdown_connections 2023-07-06T01:21:54.402 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.396+0000 7f1a3d2af700 1 --2- 172.21.15.22:0/2432558079 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a380cbc10 0x7f1a380cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.396+0000 7f1a3d2af700 1 --2- 172.21.15.22:0/2432558079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a3806a710 0x7f1a3806caf0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.396+0000 7f1a3d2af700 1 -- 172.21.15.22:0/2432558079 >> 172.21.15.22:0/2432558079 conn(0x7f1a380b1890 msgr2=0x7f1a380b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:54.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.396+0000 7f1a3d2af700 1 -- 172.21.15.22:0/2432558079 shutdown_connections 2023-07-06T01:21:54.403 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.396+0000 7f1a3d2af700 1 -- 172.21.15.22:0/2432558079 wait complete. 2023-07-06T01:21:54.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.397+0000 7f1a3d2af700 1 Processor -- start 2023-07-06T01:21:54.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.397+0000 7f1a3d2af700 1 -- start start 2023-07-06T01:21:54.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.398+0000 7f1a3d2af700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a3806a710 0x7f1a381f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.406 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.398+0000 7f1a3d2af700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a380af2a0 0x7f1a3815f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.398+0000 7f1a3d2af700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a380cbc10 0x7f1a380ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.398+0000 7f1a3d2af700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a380b46b0 con 0x7f1a3806a710 2023-07-06T01:21:54.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.398+0000 7f1a3d2af700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a380b3d10 con 0x7f1a380cbc10 2023-07-06T01:21:54.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.398+0000 7f1a3d2af700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a380b3ff0 con 0x7f1a380af2a0 2023-07-06T01:21:54.407 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.398+0000 7f1a36ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a3806a710 0x7f1a381f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.398+0000 7f1a36ffd700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a3806a710 0x7f1a381f2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48368/0 (socket says 172.21.15.22:48368) 2023-07-06T01:21:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.398+0000 7f1a36ffd700 1 -- 172.21.15.22:0/1137555264 learned_addr learned my addr 172.21.15.22:0/1137555264 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.399+0000 7f1a377fe700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a380cbc10 0x7f1a380ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.408 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.399+0000 7f1a367fc700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a380af2a0 0x7f1a3815f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.399+0000 7f1a36ffd700 1 -- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a380af2a0 msgr2=0x7f1a3815f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.399+0000 7f1a36ffd700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a380af2a0 0x7f1a3815f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.399+0000 7f1a36ffd700 1 -- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a380cbc10 msgr2=0x7f1a380ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.409 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.399+0000 7f1a36ffd700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a380cbc10 0x7f1a380ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.399+0000 7f1a36ffd700 1 -- 172.21.15.22:0/1137555264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c009550 con 0x7f1a3806a710 2023-07-06T01:21:54.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.399+0000 7f1a36ffd700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a3806a710 0x7f1a381f2ad0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800c4d0 tx=0x7f1a2800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.399+0000 7f1a1f7fe700 1 -- 172.21.15.22:0/1137555264 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28011790 con 0x7f1a3806a710 2023-07-06T01:21:54.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.399+0000 7f1a1f7fe700 1 -- 172.21.15.22:0/1137555264 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a280118f0 con 0x7f1a3806a710 2023-07-06T01:21:54.410 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.399+0000 7f1a1f7fe700 1 -- 172.21.15.22:0/1137555264 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28011ca0 con 0x7f1a3806a710 2023-07-06T01:21:54.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.399+0000 7f1a3d2af700 1 -- 172.21.15.22:0/1137555264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a380b42d0 con 0x7f1a3806a710 2023-07-06T01:21:54.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.399+0000 7f1a3d2af700 1 -- 172.21.15.22:0/1137555264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a381f3010 con 0x7f1a3806a710 2023-07-06T01:21:54.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.400+0000 7f1a3d2af700 1 -- 172.21.15.22:0/1137555264 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a380b47f0 con 0x7f1a3806a710 2023-07-06T01:21:54.411 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.400+0000 7f1a1f7fe700 1 -- 172.21.15.22:0/1137555264 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a2801c460 con 0x7f1a3806a710 2023-07-06T01:21:54.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.401+0000 7f1a1f7fe700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1a2406ace0 0x7f1a2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.401+0000 7f1a1f7fe700 1 -- 172.21.15.22:0/1137555264 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1a28088fa0 con 0x7f1a3806a710 2023-07-06T01:21:54.412 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.401+0000 7f1a367fc700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1a2406ace0 0x7f1a2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.401+0000 7f1a367fc700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1a2406ace0 0x7f1a2406d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0098f0 tx=0x7f1a2c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.413 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.405+0000 7f1a1f7fe700 1 -- 172.21.15.22:0/1137555264 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1a280bb310 con 0x7f1a3806a710 2023-07-06T01:21:54.603 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.602+0000 7f1a3d2af700 1 -- 172.21.15.22:0/1137555264 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a380b3e80 con 0x7f1a2406ace0 2023-07-06T01:21:54.604 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.602+0000 7f1a1f7fe700 1 -- 172.21.15.22:0/1137555264 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1a380b3e80 con 0x7f1a2406ace0 2023-07-06T01:21:54.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.604+0000 7f1a3d2af700 1 -- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1a2406ace0 msgr2=0x7f1a2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.606 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.604+0000 7f1a3d2af700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1a2406ace0 0x7f1a2406d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0098f0 tx=0x7f1a2c008030 comp rx=0 tx=0).stop 2023-07-06T01:21:54.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.604+0000 7f1a3d2af700 1 -- 172.21.15.22:0/1137555264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a3806a710 msgr2=0x7f1a381f2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.604+0000 7f1a3d2af700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a3806a710 0x7f1a381f2ad0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800c4d0 tx=0x7f1a2800c890 comp rx=0 tx=0).stop 2023-07-06T01:21:54.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.604+0000 7f1a3d2af700 1 -- 172.21.15.22:0/1137555264 shutdown_connections 2023-07-06T01:21:54.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.604+0000 7f1a3d2af700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1a2406ace0 0x7f1a2406d190 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.607 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.604+0000 7f1a3d2af700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1a380cbc10 0x7f1a380ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.604+0000 7f1a3d2af700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a3806a710 0x7f1a381f2ad0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.604+0000 7f1a3d2af700 1 --2- 172.21.15.22:0/1137555264 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1a380af2a0 0x7f1a3815f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.604+0000 7f1a3d2af700 1 -- 172.21.15.22:0/1137555264 >> 172.21.15.22:0/1137555264 conn(0x7f1a380b1890 msgr2=0x7f1a3815c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:54.608 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.604+0000 7f1a3d2af700 1 -- 172.21.15.22:0/1137555264 shutdown_connections 2023-07-06T01:21:54.609 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.604+0000 7f1a3d2af700 1 -- 172.21.15.22:0/1137555264 wait complete. 2023-07-06T01:21:54.609 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 5 to deep-scrub 2023-07-06T01:21:54.618 INFO:tasks.ceph:Scrubbing osd.6 2023-07-06T01:21:54.619 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-07-06T01:21:54.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.746+0000 7f9b218e5700 1 Processor -- start 2023-07-06T01:21:54.748 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.746+0000 7f9b218e5700 1 -- start start 2023-07-06T01:21:54.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b218e5700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9b1c0b3b70 0x7f9b1c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.750 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b218e5700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9b1c0b40c0 0x7f9b1c0d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b218e5700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1c0cebd0 con 0x7f9b1c0b36f0 2023-07-06T01:21:54.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b218e5700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1c15cff0 con 0x7f9b1c0b40c0 2023-07-06T01:21:54.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b218e5700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1c0cdfc0 con 0x7f9b1c0b3b70 2023-07-06T01:21:54.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b1b7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9b1c0b40c0 0x7f9b1c0d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b1b7fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9b1c0b40c0 0x7f9b1c0d0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48380/0 (socket says 172.21.15.22:48380) 2023-07-06T01:21:54.751 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b1b7fe700 1 -- 172.21.15.22:0/2494776214 learned_addr learned my addr 172.21.15.22:0/2494776214 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:54.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b1a7fc700 1 --2- 172.21.15.22:0/2494776214 >> v2:172.21.15.103:3300/0 conn(0x7f9b1c0b3b70 0x7f9b1c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b1b7fe700 1 -- 172.21.15.22:0/2494776214 >> v2:172.21.15.103:3300/0 conn(0x7f9b1c0b3b70 msgr2=0x7f9b1c0d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:54.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b1b7fe700 1 --2- 172.21.15.22:0/2494776214 >> v2:172.21.15.103:3300/0 conn(0x7f9b1c0b3b70 0x7f9b1c0d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.752 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b1b7fe700 1 -- 172.21.15.22:0/2494776214 >> v1:172.21.15.103:6789/0 conn(0x7f9b1c0b36f0 legacy=0x7f9b1c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:54.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b1b7fe700 1 -- 172.21.15.22:0/2494776214 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c0ce2a0 con 0x7f9b1c0b40c0 2023-07-06T01:21:54.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b1a7fc700 1 --2- 172.21.15.22:0/2494776214 >> v2:172.21.15.103:3300/0 conn(0x7f9b1c0b3b70 0x7f9b1c0d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:54.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b1b7fe700 1 --2- 172.21.15.22:0/2494776214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b1c0b40c0 0x7f9b1c0d0fd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c08d280 tx=0x7f9b1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea3265e57cbd3e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b19ffb700 1 -- 172.21.15.22:0/2494776214 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1400e070 con 0x7f9b1c0b40c0 2023-07-06T01:21:54.753 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b19ffb700 1 -- 172.21.15.22:0/2494776214 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b14003a20 con 0x7f9b1c0b40c0 2023-07-06T01:21:54.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.747+0000 7f9b19ffb700 1 -- 172.21.15.22:0/2494776214 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14003dd0 con 0x7f9b1c0b40c0 2023-07-06T01:21:54.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.748+0000 7f9b218e5700 1 -- 172.21.15.22:0/2494776214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b1c0b40c0 msgr2=0x7f9b1c0d0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.748+0000 7f9b218e5700 1 --2- 172.21.15.22:0/2494776214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b1c0b40c0 0x7f9b1c0d0fd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c08d280 tx=0x7f9b1400b330 comp rx=0 tx=0).stop 2023-07-06T01:21:54.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.748+0000 7f9b218e5700 1 -- 172.21.15.22:0/2494776214 shutdown_connections 2023-07-06T01:21:54.754 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.748+0000 7f9b218e5700 1 --2- 172.21.15.22:0/2494776214 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b1c0b40c0 0x7f9b1c0d0fd0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.748+0000 7f9b218e5700 1 --2- 172.21.15.22:0/2494776214 >> v2:172.21.15.103:3300/0 conn(0x7f9b1c0b3b70 0x7f9b1c0d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.748+0000 7f9b218e5700 1 -- 172.21.15.22:0/2494776214 >> 172.21.15.22:0/2494776214 conn(0x7f9b1c09c2e0 msgr2=0x7f9b1c0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:54.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.748+0000 7f9b218e5700 1 -- 172.21.15.22:0/2494776214 shutdown_connections 2023-07-06T01:21:54.755 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.748+0000 7f9b218e5700 1 -- 172.21.15.22:0/2494776214 wait complete. 2023-07-06T01:21:54.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.749+0000 7f9b218e5700 1 Processor -- start 2023-07-06T01:21:54.758 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.749+0000 7f9b218e5700 1 -- start start 2023-07-06T01:21:54.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.750+0000 7f9b218e5700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c0b36f0 0x7f9b1c0afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.750+0000 7f9b218e5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b1c0b3b70 0x7f9b1c0b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.750+0000 7f9b218e5700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b1c0b40c0 0x7f9b1c1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.759 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.750+0000 7f9b218e5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c158b10 con 0x7f9b1c0b3b70 2023-07-06T01:21:54.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.750+0000 7f9b218e5700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c0b0120 con 0x7f9b1c0b36f0 2023-07-06T01:21:54.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.750+0000 7f9b218e5700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c0b0400 con 0x7f9b1c0b40c0 2023-07-06T01:21:54.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.750+0000 7f9b1a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b1c0b3b70 0x7f9b1c0b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.760 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.750+0000 7f9b1a7fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b1c0b3b70 0x7f9b1c0b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48384/0 (socket says 172.21.15.22:48384) 2023-07-06T01:21:54.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.750+0000 7f9b1a7fc700 1 -- 172.21.15.22:0/3162737733 learned_addr learned my addr 172.21.15.22:0/3162737733 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:54.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.750+0000 7f9b1b7fe700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b1c0b40c0 0x7f9b1c1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b1affd700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c0b36f0 0x7f9b1c0afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b1a7fc700 1 -- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b1c0b40c0 msgr2=0x7f9b1c1585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.761 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b1a7fc700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b1c0b40c0 0x7f9b1c1585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b1a7fc700 1 -- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c0b36f0 msgr2=0x7f9b1c0afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b1a7fc700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c0b36f0 0x7f9b1c0afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b1a7fc700 1 -- 172.21.15.22:0/3162737733 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b14009b30 con 0x7f9b1c0b3b70 2023-07-06T01:21:54.762 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b1affd700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b1c0b36f0 0x7f9b1c0afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:54.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b1b7fe700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b1c0b40c0 0x7f9b1c1585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:54.763 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b1a7fc700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b1c0b3b70 0x7f9b1c0b25e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00c750 tx=0x7f9b0c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b03fff700 1 -- 172.21.15.22:0/3162737733 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c011800 con 0x7f9b1c0b3b70 2023-07-06T01:21:54.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b03fff700 1 -- 172.21.15.22:0/3162737733 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c011960 con 0x7f9b1c0b3b70 2023-07-06T01:21:54.764 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b218e5700 1 -- 172.21.15.22:0/3162737733 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1c0b06e0 con 0x7f9b1c0b3b70 2023-07-06T01:21:54.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b03fff700 1 -- 172.21.15.22:0/3162737733 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c011cd0 con 0x7f9b1c0b3b70 2023-07-06T01:21:54.765 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.751+0000 7f9b218e5700 1 -- 172.21.15.22:0/3162737733 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c0b0ca0 con 0x7f9b1c0b3b70 2023-07-06T01:21:54.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.752+0000 7f9b218e5700 1 -- 172.21.15.22:0/3162737733 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9b1c0ce860 con 0x7f9b1c0b3b70 2023-07-06T01:21:54.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.752+0000 7f9b03fff700 1 -- 172.21.15.22:0/3162737733 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9b0c021350 con 0x7f9b1c0b3b70 2023-07-06T01:21:54.766 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.753+0000 7f9b03fff700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9b0806ace0 0x7f9b0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.753+0000 7f9b03fff700 1 -- 172.21.15.22:0/3162737733 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9b0c089180 con 0x7f9b1c0b3b70 2023-07-06T01:21:54.767 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.753+0000 7f9b03fff700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] conn(0x7f9b08071150 0x7f9b08073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.753+0000 7f9b03fff700 1 -- 172.21.15.22:0/3162737733 --> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b08073c30 con 0x7f9b08071150 2023-07-06T01:21:54.768 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.753+0000 7f9b03fff700 1 -- 172.21.15.22:0/3162737733 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b0c08b820 con 0x7f9b1c0b3b70 2023-07-06T01:21:54.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.753+0000 7f9b1affd700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9b0806ace0 0x7f9b0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.753+0000 7f9b1b7fe700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] conn(0x7f9b08071150 0x7f9b08073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.769 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.754+0000 7f9b1affd700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9b0806ace0 0x7f9b0806d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9b10006fd0 tx=0x7f9b10008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.770 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.754+0000 7f9b1b7fe700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] conn(0x7f9b08071150 0x7f9b08073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.755+0000 7f9b03fff700 1 -- 172.21.15.22:0/3162737733 <== osd.6 v2:172.21.15.103:6808/2564178583 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9b08073c30 con 0x7f9b08071150 2023-07-06T01:21:54.771 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.769+0000 7f9b218e5700 1 -- 172.21.15.22:0/3162737733 --> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9b1c0cb320 con 0x7f9b08071150 2023-07-06T01:21:54.772 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.769+0000 7f9b03fff700 1 -- 172.21.15.22:0/3162737733 <== osd.6 v2:172.21.15.103:6808/2564178583 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9b1c0cb320 con 0x7f9b08071150 2023-07-06T01:21:54.772 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:21:54.772 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T01:21:54.773 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:21:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.769+0000 7f9b218e5700 1 -- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] conn(0x7f9b08071150 msgr2=0x7f9b08073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.769+0000 7f9b218e5700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:6808/2564178583,v1:172.21.15.103:6809/2564178583] conn(0x7f9b08071150 0x7f9b08073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.769+0000 7f9b218e5700 1 -- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9b0806ace0 msgr2=0x7f9b0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.769+0000 7f9b218e5700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f9b0806ace0 0x7f9b0806d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9b10006fd0 tx=0x7f9b10008040 comp rx=0 tx=0).stop 2023-07-06T01:21:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.770+0000 7f9b218e5700 1 -- 172.21.15.22:0/3162737733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b1c0b3b70 msgr2=0x7f9b1c0b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.770+0000 7f9b218e5700 1 --2- 172.21.15.22:0/3162737733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b1c0b3b70 0x7f9b1c0b25e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00c750 tx=0x7f9b0c00cb10 comp rx=0 tx=0).stop 2023-07-06T01:21:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.770+0000 7f9b1b7fe700 1 -- 172.21.15.22:0/3162737733 reap_dead start 2023-07-06T01:21:54.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.770+0000 7f9b218e5700 1 -- 172.21.15.22:0/3162737733 shutdown_connections 2023-07-06T01:21:54.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.770+0000 7f9b218e5700 1 -- 172.21.15.22:0/3162737733 >> 172.21.15.22:0/3162737733 conn(0x7f9b1c09c2e0 msgr2=0x7f9b1c1e52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:54.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.770+0000 7f9b218e5700 1 -- 172.21.15.22:0/3162737733 shutdown_connections 2023-07-06T01:21:54.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.770+0000 7f9b218e5700 1 -- 172.21.15.22:0/3162737733 wait complete. 2023-07-06T01:21:54.784 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-07-06T01:21:54.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.908+0000 7fee7a79b700 1 Processor -- start 2023-07-06T01:21:54.910 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.908+0000 7fee7a79b700 1 -- start start 2023-07-06T01:21:54.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee7a79b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee7415df70 0x7fee740cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.911 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee7a79b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fee740d0130 0x7fee740ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee7a79b700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee740ce550 con 0x7fee74159870 2023-07-06T01:21:54.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee7a79b700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee740d05b0 con 0x7fee740d0130 2023-07-06T01:21:54.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee7a79b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee740cdb60 con 0x7fee7415df70 2023-07-06T01:21:54.912 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee78d38700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fee740d0130 0x7fee740ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee78d38700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fee740d0130 0x7fee740ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48396/0 (socket says 172.21.15.22:48396) 2023-07-06T01:21:54.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee78d38700 1 -- 172.21.15.22:0/3316644166 learned_addr learned my addr 172.21.15.22:0/3316644166 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:54.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee737fe700 1 --2- 172.21.15.22:0/3316644166 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee7415df70 0x7fee740cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee78d38700 1 -- 172.21.15.22:0/3316644166 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee7415df70 msgr2=0x7fee740cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:54.913 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee78d38700 1 --2- 172.21.15.22:0/3316644166 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee7415df70 0x7fee740cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee78d38700 1 -- 172.21.15.22:0/3316644166 >> v1:172.21.15.103:6789/0 conn(0x7fee74159870 legacy=0x7fee740715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee78d38700 1 -- 172.21.15.22:0/3316644166 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7415d490 con 0x7fee740d0130 2023-07-06T01:21:54.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.909+0000 7fee78d38700 1 --2- 172.21.15.22:0/3316644166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee740d0130 0x7fee740ca360 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009e50 tx=0x7fee6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b58bd39b0afbcc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.910+0000 7fee72ffd700 1 -- 172.21.15.22:0/3316644166 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c00e070 con 0x7fee740d0130 2023-07-06T01:21:54.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.910+0000 7fee72ffd700 1 -- 172.21.15.22:0/3316644166 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c00b910 con 0x7fee740d0130 2023-07-06T01:21:54.914 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.910+0000 7fee72ffd700 1 -- 172.21.15.22:0/3316644166 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c00bcc0 con 0x7fee740d0130 2023-07-06T01:21:54.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.910+0000 7fee7a79b700 1 -- 172.21.15.22:0/3316644166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee740d0130 msgr2=0x7fee740ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.910+0000 7fee7a79b700 1 --2- 172.21.15.22:0/3316644166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee740d0130 0x7fee740ca360 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009e50 tx=0x7fee6c00b330 comp rx=0 tx=0).stop 2023-07-06T01:21:54.917 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.910+0000 7fee7a79b700 1 -- 172.21.15.22:0/3316644166 shutdown_connections 2023-07-06T01:21:54.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.910+0000 7fee7a79b700 1 --2- 172.21.15.22:0/3316644166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee740d0130 0x7fee740ca360 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.910+0000 7fee7a79b700 1 --2- 172.21.15.22:0/3316644166 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee7415df70 0x7fee740cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.910+0000 7fee7a79b700 1 -- 172.21.15.22:0/3316644166 >> 172.21.15.22:0/3316644166 conn(0x7fee7406b110 msgr2=0x7fee7406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:54.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.910+0000 7fee7a79b700 1 -- 172.21.15.22:0/3316644166 shutdown_connections 2023-07-06T01:21:54.918 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.910+0000 7fee7a79b700 1 -- 172.21.15.22:0/3316644166 wait complete. 2023-07-06T01:21:54.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.911+0000 7fee7a79b700 1 Processor -- start 2023-07-06T01:21:54.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.912+0000 7fee7a79b700 1 -- start start 2023-07-06T01:21:54.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.912+0000 7fee7a79b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee740d0130 0x7fee740ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.912+0000 7fee7a79b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee74159870 0x7fee740cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.919 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.912+0000 7fee7a79b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee7415df70 0x7fee740cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.912+0000 7fee7a79b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee740cad90 con 0x7fee7415df70 2023-07-06T01:21:54.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.912+0000 7fee7a79b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee740cbdb0 con 0x7fee740d0130 2023-07-06T01:21:54.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.912+0000 7fee7a79b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee740cc090 con 0x7fee74159870 2023-07-06T01:21:54.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.912+0000 7fee78d38700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee7415df70 0x7fee740cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.920 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee78d38700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee7415df70 0x7fee740cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48402/0 (socket says 172.21.15.22:48402) 2023-07-06T01:21:54.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee78d38700 1 -- 172.21.15.22:0/2840320800 learned_addr learned my addr 172.21.15.22:0/2840320800 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:54.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee737fe700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee74159870 0x7fee740cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee73fff700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee740d0130 0x7fee740ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee78d38700 1 -- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee74159870 msgr2=0x7fee740cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.921 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee78d38700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee74159870 0x7fee740cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee78d38700 1 -- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee740d0130 msgr2=0x7fee740ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:54.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee78d38700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee740d0130 0x7fee740ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:54.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee78d38700 1 -- 172.21.15.22:0/2840320800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c009b30 con 0x7fee7415df70 2023-07-06T01:21:54.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee78d38700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee7415df70 0x7fee740cb720 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fee7406c600 tx=0x7fee6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.922 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee70ff9700 1 -- 172.21.15.22:0/2840320800 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c004960 con 0x7fee7415df70 2023-07-06T01:21:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee70ff9700 1 -- 172.21.15.22:0/2840320800 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c0099b0 con 0x7fee7415df70 2023-07-06T01:21:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee7a79b700 1 -- 172.21.15.22:0/2840320800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee740cc370 con 0x7fee7415df70 2023-07-06T01:21:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee70ff9700 1 -- 172.21.15.22:0/2840320800 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c01c680 con 0x7fee7415df70 2023-07-06T01:21:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.913+0000 7fee7a79b700 1 -- 172.21.15.22:0/2840320800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7414be70 con 0x7fee7415df70 2023-07-06T01:21:54.923 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.914+0000 7fee7a79b700 1 -- 172.21.15.22:0/2840320800 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee74071d20 con 0x7fee7415df70 2023-07-06T01:21:54.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.915+0000 7fee70ff9700 1 -- 172.21.15.22:0/2840320800 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fee6c01c7e0 con 0x7fee7415df70 2023-07-06T01:21:54.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.915+0000 7fee70ff9700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee5406ace0 0x7fee5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:54.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.915+0000 7fee70ff9700 1 -- 172.21.15.22:0/2840320800 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fee6c08ab60 con 0x7fee7415df70 2023-07-06T01:21:54.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.915+0000 7fee73fff700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee5406ace0 0x7fee5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:54.924 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.916+0000 7fee73fff700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee5406ace0 0x7fee5406d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000f60 tx=0x7fee5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:54.925 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:54.919+0000 7fee70ff9700 1 -- 172.21.15.22:0/2840320800 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fee6c055f10 con 0x7fee7415df70 2023-07-06T01:21:55.116 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.114+0000 7fee7a79b700 1 -- 172.21.15.22:0/2840320800 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fee7415da50 con 0x7fee5406ace0 2023-07-06T01:21:55.117 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.115+0000 7fee70ff9700 1 -- 172.21.15.22:0/2840320800 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fee5c0050d0 con 0x7fee5406ace0 2023-07-06T01:21:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.117+0000 7fee7a79b700 1 -- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee5406ace0 msgr2=0x7fee5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.117+0000 7fee7a79b700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee5406ace0 0x7fee5406d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000f60 tx=0x7fee5c009040 comp rx=0 tx=0).stop 2023-07-06T01:21:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.117+0000 7fee7a79b700 1 -- 172.21.15.22:0/2840320800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee7415df70 msgr2=0x7fee740cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.119 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.117+0000 7fee7a79b700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee7415df70 0x7fee740cb720 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fee7406c600 tx=0x7fee6c004ad0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.117+0000 7fee7a79b700 1 -- 172.21.15.22:0/2840320800 shutdown_connections 2023-07-06T01:21:55.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.117+0000 7fee7a79b700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee7415df70 0x7fee740cb720 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.117+0000 7fee7a79b700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fee5406ace0 0x7fee5406d190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.120 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.117+0000 7fee7a79b700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee740d0130 0x7fee740ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.117+0000 7fee7a79b700 1 --2- 172.21.15.22:0/2840320800 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee74159870 0x7fee740cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.117+0000 7fee7a79b700 1 -- 172.21.15.22:0/2840320800 >> 172.21.15.22:0/2840320800 conn(0x7fee7406b110 msgr2=0x7fee7414e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:55.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.117+0000 7fee7a79b700 1 -- 172.21.15.22:0/2840320800 shutdown_connections 2023-07-06T01:21:55.121 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.117+0000 7fee7a79b700 1 -- 172.21.15.22:0/2840320800 wait complete. 2023-07-06T01:21:55.122 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 6 to deep-scrub 2023-07-06T01:21:55.131 INFO:tasks.ceph:Scrubbing osd.7 2023-07-06T01:21:55.131 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-07-06T01:21:55.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.259+0000 7ff8bd7ab700 1 Processor -- start 2023-07-06T01:21:55.261 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.259+0000 7ff8bd7ab700 1 -- start start 2023-07-06T01:21:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8bd7ab700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff8b80b36f0 0x7ff8b80cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.263 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8bd7ab700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff8b80cf5a0 0x7ff8b80d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8bd7ab700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b80cebd0 con 0x7ff8b80b3b70 2023-07-06T01:21:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8bd7ab700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b815cff0 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8bd7ab700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b80cdfc0 con 0x7ff8b80b36f0 2023-07-06T01:21:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8b67fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff8b80cf5a0 0x7ff8b80d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.264 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8b67fc700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7ff8b80cf5a0 0x7ff8b80d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48406/0 (socket says 172.21.15.22:48406) 2023-07-06T01:21:55.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8b67fc700 1 -- 172.21.15.22:0/1825358771 learned_addr learned my addr 172.21.15.22:0/1825358771 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:55.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8b6ffd700 1 --2- 172.21.15.22:0/1825358771 >> v2:172.21.15.103:3300/0 conn(0x7ff8b80b36f0 0x7ff8b80cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8b67fc700 1 -- 172.21.15.22:0/1825358771 >> v2:172.21.15.103:3300/0 conn(0x7ff8b80b36f0 msgr2=0x7ff8b80cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:55.265 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8b67fc700 1 --2- 172.21.15.22:0/1825358771 >> v2:172.21.15.103:3300/0 conn(0x7ff8b80b36f0 0x7ff8b80cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8b67fc700 1 -- 172.21.15.22:0/1825358771 >> v1:172.21.15.103:6789/0 conn(0x7ff8b80b3b70 legacy=0x7ff8b80b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8b67fc700 1 -- 172.21.15.22:0/1825358771 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b80ce2a0 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8b6ffd700 1 --2- 172.21.15.22:0/1825358771 >> v2:172.21.15.103:3300/0 conn(0x7ff8b80b36f0 0x7ff8b80cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:21:55.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.260+0000 7ff8b67fc700 1 --2- 172.21.15.22:0/1825358771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b80cf5a0 0x7ff8b80d0a90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff8b808b8d0 tx=0x7ff8a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef323844d7535a70 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:55.266 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.261+0000 7ff8b5ffb700 1 -- 172.21.15.22:0/1825358771 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a800b870 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.261+0000 7ff8b5ffb700 1 -- 172.21.15.22:0/1825358771 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a800b9d0 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.261+0000 7ff8b5ffb700 1 -- 172.21.15.22:0/1825358771 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a800bd60 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.261+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/1825358771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b80cf5a0 msgr2=0x7ff8b80d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.267 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.261+0000 7ff8bd7ab700 1 --2- 172.21.15.22:0/1825358771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b80cf5a0 0x7ff8b80d0a90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff8b808b8d0 tx=0x7ff8a8009bd0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.261+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/1825358771 shutdown_connections 2023-07-06T01:21:55.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.261+0000 7ff8bd7ab700 1 --2- 172.21.15.22:0/1825358771 >> v2:172.21.15.103:3300/0 conn(0x7ff8b80b36f0 0x7ff8b80cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.261+0000 7ff8bd7ab700 1 --2- 172.21.15.22:0/1825358771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b80cf5a0 0x7ff8b80d0a90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.261+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/1825358771 >> 172.21.15.22:0/1825358771 conn(0x7ff8b809c2e0 msgr2=0x7ff8b80d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:55.268 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.261+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/1825358771 shutdown_connections 2023-07-06T01:21:55.269 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.261+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/1825358771 wait complete. 2023-07-06T01:21:55.271 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.263+0000 7ff8bd7ab700 1 Processor -- start 2023-07-06T01:21:55.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.263+0000 7ff8bd7ab700 1 -- start start 2023-07-06T01:21:55.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.263+0000 7ff8bd7ab700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8b80b36f0 0x7ff8b8159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.263+0000 7ff8bd7ab700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff8b80b3b70 0x7ff8b815a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.263+0000 7ff8bd7ab700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b80cf5a0 0x7ff8b815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.272 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.263+0000 7ff8bd7ab700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b80c6990 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.263+0000 7ff8bd7ab700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b81e46a0 con 0x7ff8b80b36f0 2023-07-06T01:21:55.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.263+0000 7ff8bd7ab700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8159760 con 0x7ff8b80b3b70 2023-07-06T01:21:55.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.263+0000 7ff8b77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b80cf5a0 0x7ff8b815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.263+0000 7ff8b77fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b80cf5a0 0x7ff8b815b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48414/0 (socket says 172.21.15.22:48414) 2023-07-06T01:21:55.273 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.264+0000 7ff8b77fe700 1 -- 172.21.15.22:0/3535204672 learned_addr learned my addr 172.21.15.22:0/3535204672 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.264+0000 7ff8b67fc700 1 --2- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff8b80b3b70 0x7ff8b815a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.264+0000 7ff8b77fe700 1 -- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff8b80b3b70 msgr2=0x7ff8b815a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.264+0000 7ff8b77fe700 1 --2- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff8b80b3b70 0x7ff8b815a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.264+0000 7ff8b77fe700 1 -- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8b80b36f0 msgr2=0x7ff8b8159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.264+0000 7ff8b77fe700 1 --2- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8b80b36f0 0x7ff8b8159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.264+0000 7ff8b77fe700 1 -- 172.21.15.22:0/3535204672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a8009550 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.264+0000 7ff8b77fe700 1 --2- 172.21.15.22:0/3535204672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b80cf5a0 0x7ff8b815b3c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000cc80 tx=0x7ff8b000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.264+0000 7ff89f7fe700 1 -- 172.21.15.22:0/3535204672 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0003c00 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.264+0000 7ff89f7fe700 1 -- 172.21.15.22:0/3535204672 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b0003d60 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.275 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.264+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/3535204672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b80c6be0 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.264+0000 7ff89f7fe700 1 -- 172.21.15.22:0/3535204672 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b000ab30 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.264+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/3535204672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b80c71a0 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.265+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/3535204672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8b80ce860 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.266+0000 7ff89f7fe700 1 -- 172.21.15.22:0/3535204672 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff8b000ac90 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.276 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.266+0000 7ff89f7fe700 1 --2- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff8a406ad10 0x7ff8a406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.266+0000 7ff89f7fe700 1 -- 172.21.15.22:0/3535204672 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff8b0089b80 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.267+0000 7ff89f7fe700 1 --2- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] conn(0x7ff8a4071180 0x7ff8a4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.267+0000 7ff8b6ffd700 1 --2- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff8a406ad10 0x7ff8a406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.267+0000 7ff89f7fe700 1 -- 172.21.15.22:0/3535204672 --> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8a4073c60 con 0x7ff8a4071180 2023-07-06T01:21:55.277 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.267+0000 7ff89f7fe700 1 -- 172.21.15.22:0/3535204672 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8b0089fb0 con 0x7ff8b80cf5a0 2023-07-06T01:21:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.267+0000 7ff8b67fc700 1 --2- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] conn(0x7ff8a4071180 0x7ff8a4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.267+0000 7ff8b6ffd700 1 --2- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff8a406ad10 0x7ff8a406d1c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac006fd0 tx=0x7ff8ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.267+0000 7ff8b67fc700 1 --2- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] conn(0x7ff8a4071180 0x7ff8a4073590 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.268+0000 7ff89f7fe700 1 -- 172.21.15.22:0/3535204672 <== osd.7 v2:172.21.15.103:6816/1845979485 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff8a4073c60 con 0x7ff8a4071180 2023-07-06T01:21:55.284 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.282+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/3535204672 --> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff8b80ce580 con 0x7ff8a4071180 2023-07-06T01:21:55.285 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.283+0000 7ff89f7fe700 1 -- 172.21.15.22:0/3535204672 <== osd.7 v2:172.21.15.103:6816/1845979485 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff8a8021040 con 0x7ff8a4071180 2023-07-06T01:21:55.285 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2023-07-06T01:21:55.285 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T01:21:55.286 INFO:teuthology.orchestra.run.smithi022.stdout:} 2023-07-06T01:21:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.283+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] conn(0x7ff8a4071180 msgr2=0x7ff8a4073590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.283+0000 7ff8bd7ab700 1 --2- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:6816/1845979485,v1:172.21.15.103:6817/1845979485] conn(0x7ff8a4071180 0x7ff8a4073590 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.283+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff8a406ad10 msgr2=0x7ff8a406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.283+0000 7ff8bd7ab700 1 --2- 172.21.15.22:0/3535204672 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7ff8a406ad10 0x7ff8a406d1c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac006fd0 tx=0x7ff8ac008040 comp rx=0 tx=0).stop 2023-07-06T01:21:55.287 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.283+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/3535204672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b80cf5a0 msgr2=0x7ff8b815b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.283+0000 7ff8bd7ab700 1 --2- 172.21.15.22:0/3535204672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff8b80cf5a0 0x7ff8b815b3c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000cc80 tx=0x7ff8b000cf90 comp rx=0 tx=0).stop 2023-07-06T01:21:55.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.283+0000 7ff8b77fe700 1 -- 172.21.15.22:0/3535204672 reap_dead start 2023-07-06T01:21:55.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.283+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/3535204672 shutdown_connections 2023-07-06T01:21:55.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.283+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/3535204672 >> 172.21.15.22:0/3535204672 conn(0x7ff8b809c2e0 msgr2=0x7ff8b806c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:55.288 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.283+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/3535204672 shutdown_connections 2023-07-06T01:21:55.289 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.283+0000 7ff8bd7ab700 1 -- 172.21.15.22:0/3535204672 wait complete. 2023-07-06T01:21:55.297 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-07-06T01:21:55.423 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.421+0000 7f7cf92fe700 1 Processor -- start 2023-07-06T01:21:55.424 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.421+0000 7f7cf92fe700 1 -- start start 2023-07-06T01:21:55.425 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.422+0000 7f7cf92fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7cf415c750 0x7f7cf40cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.422+0000 7f7cf92fe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7cf40cdf80 0x7f7cf406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.422+0000 7f7cf92fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7cf40ce400 0x7f7cf406b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.422+0000 7f7cf92fe700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf40d06f0 con 0x7f7cf415c750 2023-07-06T01:21:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.422+0000 7f7cf92fe700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf40ceb90 con 0x7f7cf40cdf80 2023-07-06T01:21:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.422+0000 7f7cf92fe700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf40cee70 con 0x7f7cf40ce400 2023-07-06T01:21:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.422+0000 7f7cf2ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7cf415c750 0x7f7cf40cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.422+0000 7f7cf2ffd700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7cf415c750 0x7f7cf40cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48420/0 (socket says 172.21.15.22:48420) 2023-07-06T01:21:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.422+0000 7f7cf2ffd700 1 -- 172.21.15.22:0/1450066960 learned_addr learned my addr 172.21.15.22:0/1450066960 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.422+0000 7f7cf37fe700 1 --2- 172.21.15.22:0/1450066960 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7cf40ce400 0x7f7cf406b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf2ffd700 1 -- 172.21.15.22:0/1450066960 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7cf40ce400 msgr2=0x7f7cf406b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf2ffd700 1 --2- 172.21.15.22:0/1450066960 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7cf40ce400 0x7f7cf406b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf2ffd700 1 -- 172.21.15.22:0/1450066960 >> v2:172.21.15.103:3300/0 conn(0x7f7cf40cdf80 msgr2=0x7f7cf406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf27fc700 1 --2- 172.21.15.22:0/1450066960 >> v2:172.21.15.103:3300/0 conn(0x7f7cf40cdf80 0x7f7cf406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf2ffd700 1 --2- 172.21.15.22:0/1450066960 >> v2:172.21.15.103:3300/0 conn(0x7f7cf40cdf80 0x7f7cf406b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf2ffd700 1 -- 172.21.15.22:0/1450066960 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf40cf150 con 0x7f7cf415c750 2023-07-06T01:21:55.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf27fc700 1 --2- 172.21.15.22:0/1450066960 >> v2:172.21.15.103:3300/0 conn(0x7f7cf40cdf80 0x7f7cf406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:21:55.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf2ffd700 1 --2- 172.21.15.22:0/1450066960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf415c750 0x7f7cf40cfc30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4070f60 tx=0x7f7ce8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31024a3e56a82723 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:55.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf1ffb700 1 -- 172.21.15.22:0/1450066960 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce800b870 con 0x7f7cf415c750 2023-07-06T01:21:55.429 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf1ffb700 1 -- 172.21.15.22:0/1450066960 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce800b9d0 con 0x7f7cf415c750 2023-07-06T01:21:55.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf1ffb700 1 -- 172.21.15.22:0/1450066960 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce800bd60 con 0x7f7cf415c750 2023-07-06T01:21:55.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf92fe700 1 -- 172.21.15.22:0/1450066960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf415c750 msgr2=0x7f7cf40cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf92fe700 1 --2- 172.21.15.22:0/1450066960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf415c750 0x7f7cf40cfc30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4070f60 tx=0x7f7ce8009bd0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf92fe700 1 -- 172.21.15.22:0/1450066960 shutdown_connections 2023-07-06T01:21:55.430 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf92fe700 1 --2- 172.21.15.22:0/1450066960 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7cf40ce400 0x7f7cf406b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf92fe700 1 --2- 172.21.15.22:0/1450066960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf415c750 0x7f7cf40cfc30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf92fe700 1 --2- 172.21.15.22:0/1450066960 >> v2:172.21.15.103:3300/0 conn(0x7f7cf40cdf80 0x7f7cf406b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.423+0000 7f7cf92fe700 1 -- 172.21.15.22:0/1450066960 >> 172.21.15.22:0/1450066960 conn(0x7f7cf40999f0 msgr2=0x7f7cf4099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.424+0000 7f7cf92fe700 1 -- 172.21.15.22:0/1450066960 shutdown_connections 2023-07-06T01:21:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.424+0000 7f7cf92fe700 1 -- 172.21.15.22:0/1450066960 wait complete. 2023-07-06T01:21:55.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.425+0000 7f7cf92fe700 1 Processor -- start 2023-07-06T01:21:55.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.425+0000 7f7cf92fe700 1 -- start start 2023-07-06T01:21:55.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf92fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cf40cdf80 0x7f7cf41f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.434 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf92fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf40ce400 0x7f7cf4160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf92fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7cf415c750 0x7f7cf4159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf92fe700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf41f1f50 con 0x7f7cf40ce400 2023-07-06T01:21:55.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf92fe700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf41f6590 con 0x7f7cf40cdf80 2023-07-06T01:21:55.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf92fe700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf4160a70 con 0x7f7cf415c750 2023-07-06T01:21:55.435 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf27fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf40ce400 0x7f7cf4160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf27fc700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf40ce400 0x7f7cf4160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48428/0 (socket says 172.21.15.22:48428) 2023-07-06T01:21:55.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf27fc700 1 -- 172.21.15.22:0/4159026965 learned_addr learned my addr 172.21.15.22:0/4159026965 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:55.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf2ffd700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cf40cdf80 0x7f7cf41f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf37fe700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7cf415c750 0x7f7cf4159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.436 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf27fc700 1 -- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7cf415c750 msgr2=0x7f7cf4159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf27fc700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7cf415c750 0x7f7cf4159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf27fc700 1 -- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cf40cdf80 msgr2=0x7f7cf41f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf27fc700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cf40cdf80 0x7f7cf41f5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf27fc700 1 -- 172.21.15.22:0/4159026965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce8009550 con 0x7f7cf40ce400 2023-07-06T01:21:55.437 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf27fc700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf40ce400 0x7f7cf4160530 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400c750 tx=0x7f7ce400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:55.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cdb7fe700 1 -- 172.21.15.22:0/4159026965 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4011800 con 0x7f7cf40ce400 2023-07-06T01:21:55.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cdb7fe700 1 -- 172.21.15.22:0/4159026965 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce4011960 con 0x7f7cf40ce400 2023-07-06T01:21:55.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cdb7fe700 1 -- 172.21.15.22:0/4159026965 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4011cd0 con 0x7f7cf40ce400 2023-07-06T01:21:55.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.426+0000 7f7cf92fe700 1 -- 172.21.15.22:0/4159026965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf406f970 con 0x7f7cf40ce400 2023-07-06T01:21:55.438 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.427+0000 7f7cf92fe700 1 -- 172.21.15.22:0/4159026965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf40b0080 con 0x7f7cf40ce400 2023-07-06T01:21:55.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.428+0000 7f7cf92fe700 1 -- 172.21.15.22:0/4159026965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf40744b0 con 0x7f7cf40ce400 2023-07-06T01:21:55.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.428+0000 7f7cdb7fe700 1 -- 172.21.15.22:0/4159026965 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7ce4021350 con 0x7f7cf40ce400 2023-07-06T01:21:55.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.428+0000 7f7cdb7fe700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7ce006ace0 0x7f7ce006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.428+0000 7f7cdb7fe700 1 -- 172.21.15.22:0/4159026965 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7ce4088ff0 con 0x7f7cf40ce400 2023-07-06T01:21:55.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.429+0000 7f7cf2ffd700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7ce006ace0 0x7f7ce006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.439 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.429+0000 7f7cf2ffd700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7ce006ace0 0x7f7ce006d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009840 tx=0x7f7ce8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:55.440 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.433+0000 7f7cdb7fe700 1 -- 172.21.15.22:0/4159026965 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7ce4057ec0 con 0x7f7cf40ce400 2023-07-06T01:21:55.631 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.629+0000 7f7cf92fe700 1 -- 172.21.15.22:0/4159026965 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cf40cf710 con 0x7f7ce006ace0 2023-07-06T01:21:55.632 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.630+0000 7f7cdb7fe700 1 -- 172.21.15.22:0/4159026965 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7ce800bc40 con 0x7f7ce006ace0 2023-07-06T01:21:55.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.632+0000 7f7cf92fe700 1 -- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7ce006ace0 msgr2=0x7f7ce006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.632+0000 7f7cf92fe700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7ce006ace0 0x7f7ce006d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8009840 tx=0x7f7ce8008030 comp rx=0 tx=0).stop 2023-07-06T01:21:55.634 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.632+0000 7f7cf92fe700 1 -- 172.21.15.22:0/4159026965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf40ce400 msgr2=0x7f7cf4160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.632+0000 7f7cf92fe700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf40ce400 0x7f7cf4160530 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400c750 tx=0x7f7ce400ca60 comp rx=0 tx=0).stop 2023-07-06T01:21:55.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.632+0000 7f7cf92fe700 1 -- 172.21.15.22:0/4159026965 shutdown_connections 2023-07-06T01:21:55.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.632+0000 7f7cf92fe700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7cf415c750 0x7f7cf4159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.632+0000 7f7cf92fe700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f7ce006ace0 0x7f7ce006d190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.635 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.632+0000 7f7cf92fe700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cf40cdf80 0x7f7cf41f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.632+0000 7f7cf92fe700 1 --2- 172.21.15.22:0/4159026965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7cf40ce400 0x7f7cf4160530 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.632+0000 7f7cf92fe700 1 -- 172.21.15.22:0/4159026965 >> 172.21.15.22:0/4159026965 conn(0x7f7cf40999f0 msgr2=0x7f7cf40aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:55.636 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.632+0000 7f7cf92fe700 1 -- 172.21.15.22:0/4159026965 shutdown_connections 2023-07-06T01:21:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.632+0000 7f7cf92fe700 1 -- 172.21.15.22:0/4159026965 wait complete. 2023-07-06T01:21:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 7 to deep-scrub 2023-07-06T01:21:55.646 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:21:55.773 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.770+0000 7f6d4079e700 1 Processor -- start 2023-07-06T01:21:55.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.770+0000 7f6d4079e700 1 -- start start 2023-07-06T01:21:55.774 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.771+0000 7f6d4079e700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f6d3815a780 0x7f6d380af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.771+0000 7f6d4079e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d380cf000 0x7f6d380b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.771+0000 7f6d4079e700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d380d0ec0 con 0x7f6d380af8e0 2023-07-06T01:21:55.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.771+0000 7f6d4079e700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d380d11a0 con 0x7f6d3815a780 2023-07-06T01:21:55.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.771+0000 7f6d4079e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380d1480 con 0x7f6d380cf000 2023-07-06T01:21:55.775 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.771+0000 7f6d3e53a700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f6d3815a780 0x7f6d380af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.771+0000 7f6d3ed3b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d380cf000 0x7f6d380b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.771+0000 7f6d3e53a700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f6d3815a780 0x7f6d380af3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.22:37158/0 (socket says 172.21.15.22:37158) 2023-07-06T01:21:55.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.771+0000 7f6d3ed3b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d380cf000 0x7f6d380b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.22:59572/0 (socket says 172.21.15.22:59572) 2023-07-06T01:21:55.776 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.771+0000 7f6d3dd39700 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f6d380af8e0 0x7f6d380ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.22:41242/0 (socket says 172.21.15.22:41242) 2023-07-06T01:21:55.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.771+0000 7f6d3e53a700 1 -- 172.21.15.22:0/1345213061 learned_addr learned my addr 172.21.15.22:0/1345213061 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:55.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.771+0000 7f6d3ed3b700 1 -- 172.21.15.22:0/1345213061 >> v2:172.21.15.103:3300/0 conn(0x7f6d3815a780 msgr2=0x7f6d380af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:21:55.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.771+0000 7f6d3ed3b700 1 --2- 172.21.15.22:0/1345213061 >> v2:172.21.15.103:3300/0 conn(0x7f6d3815a780 0x7f6d380af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.772+0000 7f6d3ed3b700 1 -- 172.21.15.22:0/1345213061 >> v1:172.21.15.103:6789/0 conn(0x7f6d380af8e0 legacy=0x7f6d380ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.777 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.772+0000 7f6d3ed3b700 1 -- 172.21.15.22:0/1345213061 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380b0450 con 0x7f6d380cf000 2023-07-06T01:21:55.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.772+0000 7f6d3e53a700 1 --2- 172.21.15.22:0/1345213061 >> v2:172.21.15.103:3300/0 conn(0x7f6d3815a780 0x7f6d380af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T01:21:55.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.772+0000 7f6d3ed3b700 1 --2- 172.21.15.22:0/1345213061 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d380cf000 0x7f6d380b0970 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009d90 tx=0x7f6d3000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19726a2ce79a989 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:55.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.772+0000 7f6d3d538700 1 -- 172.21.15.22:0/1345213061 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000e070 con 0x7f6d380cf000 2023-07-06T01:21:55.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.772+0000 7f6d3d538700 1 -- 172.21.15.22:0/1345213061 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3000bb20 con 0x7f6d380cf000 2023-07-06T01:21:55.778 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.772+0000 7f6d3d538700 1 -- 172.21.15.22:0/1345213061 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d30012430 con 0x7f6d380cf000 2023-07-06T01:21:55.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.772+0000 7f6d4079e700 1 -- 172.21.15.22:0/1345213061 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d380cf000 msgr2=0x7f6d380b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.781 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.772+0000 7f6d4079e700 1 --2- 172.21.15.22:0/1345213061 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d380cf000 0x7f6d380b0970 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009d90 tx=0x7f6d3000b330 comp rx=0 tx=0).stop 2023-07-06T01:21:55.782 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.773+0000 7f6d4079e700 1 -- 172.21.15.22:0/1345213061 shutdown_connections 2023-07-06T01:21:55.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.773+0000 7f6d4079e700 1 --2- 172.21.15.22:0/1345213061 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d380cf000 0x7f6d380b0970 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.773+0000 7f6d4079e700 1 --2- 172.21.15.22:0/1345213061 >> v2:172.21.15.103:3300/0 conn(0x7f6d3815a780 0x7f6d380af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.783 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.773+0000 7f6d4079e700 1 -- 172.21.15.22:0/1345213061 >> 172.21.15.22:0/1345213061 conn(0x7f6d380999f0 msgr2=0x7f6d38099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:55.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.773+0000 7f6d4079e700 1 -- 172.21.15.22:0/1345213061 shutdown_connections 2023-07-06T01:21:55.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.773+0000 7f6d4079e700 1 -- 172.21.15.22:0/1345213061 wait complete. 2023-07-06T01:21:55.784 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.774+0000 7f6d4079e700 1 Processor -- start 2023-07-06T01:21:55.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.774+0000 7f6d4079e700 1 -- start start 2023-07-06T01:21:55.785 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d4079e700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d380af8e0 0x7f6d380c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d4079e700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d380cf000 0x7f6d380ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.786 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d4079e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3815a780 0x7f6d380b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d4079e700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380b5f60 con 0x7f6d380cf000 2023-07-06T01:21:55.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d4079e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380b6240 con 0x7f6d3815a780 2023-07-06T01:21:55.787 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d4079e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380b6520 con 0x7f6d380af8e0 2023-07-06T01:21:55.788 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d3dd39700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d380cf000 0x7f6d380ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d3e53a700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d380af8e0 0x7f6d380c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d3dd39700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d380cf000 0x7f6d380ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48442/0 (socket says 172.21.15.22:48442) 2023-07-06T01:21:55.789 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d3dd39700 1 -- 172.21.15.22:0/1812991508 learned_addr learned my addr 172.21.15.22:0/1812991508 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:21:55.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d3ed3b700 1 --2- 172.21.15.22:0/1812991508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3815a780 0x7f6d380b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d3dd39700 1 -- 172.21.15.22:0/1812991508 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d380af8e0 msgr2=0x7f6d380c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.790 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d3dd39700 1 --2- 172.21.15.22:0/1812991508 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d380af8e0 0x7f6d380c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d3dd39700 1 -- 172.21.15.22:0/1812991508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3815a780 msgr2=0x7f6d380b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.791 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d3dd39700 1 --2- 172.21.15.22:0/1812991508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3815a780 0x7f6d380b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d3dd39700 1 -- 172.21.15.22:0/1812991508 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d28009550 con 0x7f6d380cf000 2023-07-06T01:21:55.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d3dd39700 1 --2- 172.21.15.22:0/1812991508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d380cf000 0x7f6d380ca410 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400c520 tx=0x7f6d3400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:55.792 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d2effd700 1 -- 172.21.15.22:0/1812991508 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34005cb0 con 0x7f6d380cf000 2023-07-06T01:21:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d2effd700 1 -- 172.21.15.22:0/1812991508 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d34005e10 con 0x7f6d380cf000 2023-07-06T01:21:55.793 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d4079e700 1 -- 172.21.15.22:0/1812991508 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d30009a70 con 0x7f6d380cf000 2023-07-06T01:21:55.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.775+0000 7f6d2effd700 1 -- 172.21.15.22:0/1812991508 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d340072d0 con 0x7f6d380cf000 2023-07-06T01:21:55.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.776+0000 7f6d4079e700 1 -- 172.21.15.22:0/1812991508 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d38156030 con 0x7f6d380cf000 2023-07-06T01:21:55.794 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.776+0000 7f6d4079e700 1 -- 172.21.15.22:0/1812991508 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d380744b0 con 0x7f6d380cf000 2023-07-06T01:21:55.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.777+0000 7f6d2effd700 1 -- 172.21.15.22:0/1812991508 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6d34007430 con 0x7f6d380cf000 2023-07-06T01:21:55.795 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.777+0000 7f6d2effd700 1 --2- 172.21.15.22:0/1812991508 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6d1c06ace0 0x7f6d1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:21:55.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.777+0000 7f6d2effd700 1 -- 172.21.15.22:0/1812991508 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6d34088480 con 0x7f6d380cf000 2023-07-06T01:21:55.796 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.778+0000 7f6d3e53a700 1 --2- 172.21.15.22:0/1812991508 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6d1c06ace0 0x7f6d1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:21:55.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.778+0000 7f6d3e53a700 1 --2- 172.21.15.22:0/1812991508 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6d1c06ace0 0x7f6d1c06d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6d28006d30 tx=0x7f6d280092c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:21:55.797 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.782+0000 7f6d2effd700 1 -- 172.21.15.22:0/1812991508 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6d34053860 con 0x7f6d380cf000 2023-07-06T01:21:55.978 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.976+0000 7f6d4079e700 1 -- 172.21.15.22:0/1812991508 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d380b63b0 con 0x7f6d1c06ace0 2023-07-06T01:21:55.981 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.979+0000 7f6d2effd700 1 -- 172.21.15.22:0/1812991508 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73283 (secure 0 0 0) 0x7f6d380b63b0 con 0x7f6d1c06ace0 2023-07-06T01:21:55.982 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:21:55.983 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.982+0000 7f6d4079e700 1 -- 172.21.15.22:0/1812991508 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6d1c06ace0 msgr2=0x7f6d1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.982+0000 7f6d4079e700 1 --2- 172.21.15.22:0/1812991508 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6d1c06ace0 0x7f6d1c06d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6d28006d30 tx=0x7f6d280092c0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.982+0000 7f6d4079e700 1 -- 172.21.15.22:0/1812991508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d380cf000 msgr2=0x7f6d380ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:21:55.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.982+0000 7f6d4079e700 1 --2- 172.21.15.22:0/1812991508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d380cf000 0x7f6d380ca410 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400c520 tx=0x7f6d3400c830 comp rx=0 tx=0).stop 2023-07-06T01:21:55.984 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.982+0000 7f6d4079e700 1 -- 172.21.15.22:0/1812991508 shutdown_connections 2023-07-06T01:21:55.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.982+0000 7f6d4079e700 1 --2- 172.21.15.22:0/1812991508 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f6d1c06ace0 0x7f6d1c06d190 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.982+0000 7f6d4079e700 1 --2- 172.21.15.22:0/1812991508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3815a780 0x7f6d380b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.982+0000 7f6d4079e700 1 --2- 172.21.15.22:0/1812991508 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6d380af8e0 0x7f6d380c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.982+0000 7f6d4079e700 1 --2- 172.21.15.22:0/1812991508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d380cf000 0x7f6d380ca410 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:21:55.985 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.982+0000 7f6d4079e700 1 -- 172.21.15.22:0/1812991508 >> 172.21.15.22:0/1812991508 conn(0x7f6d380999f0 msgr2=0x7f6d380b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:21:55.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.982+0000 7f6d4079e700 1 -- 172.21.15.22:0/1812991508 shutdown_connections 2023-07-06T01:21:55.986 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:21:55.982+0000 7f6d4079e700 1 -- 172.21.15.22:0/1812991508 wait complete. 2023-07-06T01:21:55.986 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:21:55.996 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2023-07-06T01:21:54.036720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":313928,"kb_used_data":4872,"kb_used_omap":0,"kb_used_meta":308992,"kb_avail":754660792,"statfs":{"total":773094113280,"available":772772651008,"internally_reserved":0,"allocated":4988928,"data_stored":2801616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316407808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-2000,"num_read_kb":-2000,"num_write":-14708,"num_write_kb":-2000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-29,"num_bytes_recovered":-94,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.326615"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.752615+0000","last_change":"2023-07-06T01:19:20.074195+0000","last_active":"2023-07-06T01:21:46.752615+0000","last_peered":"2023-07-06T01:21:46.752615+0000","last_clean":"2023-07-06T01:21:46.752615+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:21:46.752615+0000","last_undegraded":"2023-07-06T01:21:46.752615+0000","last_fullsized":"2023-07-06T01:21:46.752615+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:58:56.446322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.754269+0000","last_change":"2023-07-06T01:19:19.587110+0000","last_active":"2023-07-06T01:21:46.754269+0000","last_peered":"2023-07-06T01:21:46.754269+0000","last_clean":"2023-07-06T01:21:46.754269+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:21:46.754269+0000","last_undegraded":"2023-07-06T01:21:46.754269+0000","last_fullsized":"2023-07-06T01:21:46.754269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:38.656516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":50,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.760757+0000","last_change":"2023-07-06T01:19:19.584242+0000","last_active":"2023-07-06T01:21:46.760757+0000","last_peered":"2023-07-06T01:21:46.760757+0000","last_clean":"2023-07-06T01:21:46.760757+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:21:46.760757+0000","last_undegraded":"2023-07-06T01:21:46.760757+0000","last_fullsized":"2023-07-06T01:21:46.760757+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:26:02.484123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":202,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.724661+0000","last_change":"2023-07-06T01:19:19.586495+0000","last_active":"2023-07-06T01:21:46.724661+0000","last_peered":"2023-07-06T01:21:46.724661+0000","last_clean":"2023-07-06T01:21:46.724661+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:21:46.724661+0000","last_undegraded":"2023-07-06T01:21:46.724661+0000","last_fullsized":"2023-07-06T01:21:46.724661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T05:05:38.656516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":52,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.721628+0000","last_change":"2023-07-06T01:19:19.586426+0000","last_active":"2023-07-06T01:21:46.721628+0000","last_peered":"2023-07-06T01:21:46.721628+0000","last_clean":"2023-07-06T01:21:46.721628+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:21:46.721628+0000","last_undegraded":"2023-07-06T01:21:46.721628+0000","last_fullsized":"2023-07-06T01:21:46.721628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T01:40:48.935924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":52,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.724410+0000","last_change":"2023-07-06T01:19:19.586626+0000","last_active":"2023-07-06T01:21:46.724410+0000","last_peered":"2023-07-06T01:21:46.724410+0000","last_clean":"2023-07-06T01:21:46.724410+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:21:46.724410+0000","last_undegraded":"2023-07-06T01:21:46.724410+0000","last_fullsized":"2023-07-06T01:21:46.724410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:37:27.309882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.756290+0000","last_change":"2023-07-06T01:19:19.584086+0000","last_active":"2023-07-06T01:21:46.756290+0000","last_peered":"2023-07-06T01:21:46.756290+0000","last_clean":"2023-07-06T01:21:46.756290+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:21:46.756290+0000","last_undegraded":"2023-07-06T01:21:46.756290+0000","last_fullsized":"2023-07-06T01:21:46.756290+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T09:45:03.031857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":50,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.725480+0000","last_change":"2023-07-06T01:19:19.586424+0000","last_active":"2023-07-06T01:21:46.725480+0000","last_peered":"2023-07-06T01:21:46.725480+0000","last_clean":"2023-07-06T01:21:46.725480+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:21:46.725480+0000","last_undegraded":"2023-07-06T01:21:46.725480+0000","last_fullsized":"2023-07-06T01:21:46.725480+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:08:16.360359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":50,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:46.725266+0000","last_change":"2023-07-06T01:19:19.583699+0000","last_active":"2023-07-06T01:21:46.725266+0000","last_peered":"2023-07-06T01:21:46.725266+0000","last_clean":"2023-07-06T01:21:46.725266+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:21:46.725266+0000","last_undegraded":"2023-07-06T01:21:46.725266+0000","last_fullsized":"2023-07-06T01:21:46.725266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:19:18.567022+0000","last_clean_scrub_stamp":"2023-07-06T01:19:18.567022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:16:36.301039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50140,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321700,"statfs":{"total":96636764160,"available":96585420800,"internally_reserved":0,"allocated":1069056,"data_stored":792978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46500000000000002,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.623,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.497},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48999999999999999,"15min":0.49199999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.434},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69699999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.55500000000000005,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.61499999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.504},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.42699999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72199999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.58299999999999996,"15min":0.59599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69499999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.68300000000000005,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.66100000000000003,"15min":0.67700000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71899999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.58899999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53656,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94318184,"statfs":{"total":96636764160,"available":96581820416,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68600000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.38900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68799999999999994,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.74299999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79400000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.75}]},{"osd":5,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.32800000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72199999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.5},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.68500000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.629}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79400000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64600000000000002,"15min":0.65800000000000003},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81200000000000006,"5min":0.872,"15min":0.872},"last":0.71999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36188,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335652,"statfs":{"total":96636764160,"available":96599707648,"internally_reserved":0,"allocated":1069056,"data_stored":792978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":1.2589999999999999,"15min":1.4139999999999999},"min":{"1min":0.23200000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.51300000000000001,"5min":12.23,"15min":12.23},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.312,"5min":1.5329999999999999,"15min":1.7370000000000001},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.375,"5min":15.208,"15min":15.208},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.89900000000000002,"15min":0.98499999999999999},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":1.5189999999999999,"5min":12.074,"15min":12.074},"last":0.313},{"interface":"front","average":{"1min":0.39400000000000002,"5min":1.306,"15min":1.458},"min":{"1min":0.26500000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73999999999999999,"5min":11.988,"15min":11.988},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":1.8120000000000001,"15min":2.052},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.47999999999999998,"5min":18.963000000000001,"15min":18.963000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.34799999999999998,"5min":1.5800000000000001,"15min":1.7849999999999999},"min":{"1min":0.24399999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.41299999999999998,"5min":15.244999999999999,"15min":15.244999999999999},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":1.127,"15min":1.2010000000000001},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.6080000000000001,"5min":11.923,"15min":11.923},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":1.5249999999999999,"15min":1.651},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.5950000000000002,"5min":12.019,"15min":12.019},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":1.099,"15min":1.1850000000000001},"min":{"1min":0.40100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79700000000000004,"5min":11.999000000000001,"15min":11.999000000000001},"last":0.751},{"interface":"front","average":{"1min":0.60299999999999998,"5min":1.5,"15min":1.6499999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88800000000000001,"5min":11.951000000000001,"15min":11.951000000000001},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":1.788,"15min":1.9690000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.7490000000000001,"5min":15.167999999999999,"15min":15.167999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":1.1479999999999999,"15min":1.236},"min":{"1min":0.47799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":12.041,"15min":12.041},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":1.8180000000000001,"15min":2.008},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.625,"5min":15.115,"15min":15.115},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":1.163,"15min":1.2410000000000001},"min":{"1min":0.45700000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8200000000000001,"5min":11.965,"15min":11.965},"last":0.69899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.35499999999999998,"15min":0.35999999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60199999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45400000000000001,"15min":0.45500000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.377}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66800000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0880000000000001,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.45400000000000001,"15min":0.46400000000000002},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.88800000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.45700000000000002,"15min":0.46500000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91400000000000003,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.41499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.97199999999999998,"5min":2.097,"15min":2.097},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.62},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.196,"5min":0.71699999999999997,"15min":0.63700000000000001},"min":{"1min":0.44,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":11.619,"5min":11.619,"15min":11.619},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.80600000000000005,"15min":0.73199999999999998},"min":{"1min":0.50800000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":11.603999999999999,"5min":11.603999999999999,"15min":11.603999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.93899999999999995,"15min":0.98399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.518,"5min":6.3049999999999997,"15min":6.3049999999999997},"last":0.68100000000000005}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.94799999999999995,"15min":0.99099999999999999},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.609,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":0.74399999999999999,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":11.663,"5min":11.663,"15min":11.663},"last":0.71299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.25700000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.33900000000000002},"min":{"1min":0.28799999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.371,"15min":0.36699999999999999},"min":{"1min":0.27100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.34200000000000003,"15min":0.33200000000000002},"min":{"1min":0.28899999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.42399999999999999,"15min":0.42199999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61199999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.33800000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.47899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.47099999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.41799999999999998}]},{"osd":5,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.505},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.432}]},{"osd":6,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.46200000000000002}]},{"osd":7,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54700000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.45300000000000001,"15min":0.46999999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67800000000000005,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.46200000000000002,"15min":0.47799999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.52800000000000002,"5min":1.51,"15min":1.51},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.25700000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64800000000000002,"5min":1.331,"15min":1.331},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.53400000000000003,"15min":0.56000000000000005},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.58099999999999996,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.53700000000000003,"15min":0.56100000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53600000000000003,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.376},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.53700000000000003,"15min":0.56499999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.501,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.67500000000000004,"15min":0.69499999999999995},"min":{"1min":0.44700000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.71999999999999997,"5min":1.663,"15min":1.663},"last":0.621},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.66500000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71499999999999997,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.64200000000000002,"15min":0.65800000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70199999999999996,"5min":1.667,"15min":1.667},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.68400000000000005,"15min":0.69999999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77500000000000002,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.65000000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.65700000000000003,"15min":0.67400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78100000000000003,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.78800000000000003,"15min":0.82599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76500000000000001,"5min":3.8130000000000002,"15min":3.8130000000000002},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.72399999999999998,"15min":0.753},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72599999999999998,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.75900000000000001,"15min":0.78200000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92600000000000005,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.51200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51700000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61899999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.36199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63100000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.53300000000000003,"15min":0.54400000000000004},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.61699999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.628},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73099999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.496},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81200000000000006,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.58299999999999996,"15min":0.59499999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.497},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.503,"15min":0.503},"min":{"1min":0.38200000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66700000000000004,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.374},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.33000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.64600000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.41699999999999998}]},{"osd":2,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67400000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.58699999999999997}]},{"osd":3,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.54100000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68600000000000005,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.499},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.66200000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54100000000000004,"15min":0.54600000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.437},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67800000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.63200000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71199999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.66000000000000003,"5min":0.87,"15min":0.87},"last":0.69199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:21:56.000 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-07-06T01:19:18.567022+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=19, tm_sec=18, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=21, tm_sec=50, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T01:21:56.001 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-07-06T01:19:18.567022+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=19, tm_sec=18, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=21, tm_sec=50, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T01:21:56.001 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-07-06T01:19:18.567022+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=19, tm_sec=18, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=21, tm_sec=50, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T01:21:56.001 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-07-06T01:19:18.567022+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=19, tm_sec=18, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=21, tm_sec=50, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T01:21:56.002 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-07-06T01:19:18.567022+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=19, tm_sec=18, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=21, tm_sec=50, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T01:21:56.002 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-07-06T01:19:18.567022+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=19, tm_sec=18, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=21, tm_sec=50, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T01:21:56.002 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-07-06T01:19:18.567022+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=19, tm_sec=18, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=21, tm_sec=50, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T01:21:56.003 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-07-06T01:19:18.567022+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=19, tm_sec=18, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=21, tm_sec=50, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T01:21:56.003 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-07-06T01:19:18.567022+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=19, tm_sec=18, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=1, tm_min=21, tm_sec=50, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T01:21:56.003 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-07-06T01:22:02.420 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:22:02.423 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:22:16.005 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T01:22:16.146 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.143+0000 7f1d4b2c5700 1 Processor -- start 2023-07-06T01:22:16.147 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.144+0000 7f1d4b2c5700 1 -- start start 2023-07-06T01:22:16.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d4b2c5700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1d440cc800 0x7f1d4406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d4b2c5700 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d440ce9e0 con 0x7f1d441610e0 2023-07-06T01:22:16.150 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d4b2c5700 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4406ce00 con 0x7f1d441614b0 2023-07-06T01:22:16.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d4b2c5700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4406d0e0 con 0x7f1d440cc800 2023-07-06T01:22:16.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d49862700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1d440cc800 0x7f1d4406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.151 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d49061700 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f1d441610e0 0x7f1d440ca2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:39192/0 (socket says 172.21.15.22:39192) 2023-07-06T01:22:16.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d49061700 1 -- 172.21.15.22:0/1373915377 learned_addr learned my addr 172.21.15.22:0/1373915377 (peer_addr_for_me v1:172.21.15.22:0/0) 2023-07-06T01:22:16.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d49862700 1 -- 172.21.15.22:0/1373915377 >> v1:172.21.15.103:6789/0 conn(0x7f1d441614b0 legacy=0x7f1d440ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:22:16.152 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d49862700 1 -- 172.21.15.22:0/1373915377 >> v1:172.21.15.22:6789/0 conn(0x7f1d441610e0 legacy=0x7f1d440ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d37fff700 1 -- 172.21.15.22:0/1373915377 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630214758 0 0) 0x7f1d440ce9e0 con 0x7f1d441610e0 2023-07-06T01:22:16.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d49862700 1 -- 172.21.15.22:0/1373915377 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d440cd100 con 0x7f1d440cc800 2023-07-06T01:22:16.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d49862700 1 --2- 172.21.15.22:0/1373915377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d440cc800 0x7f1d4406d2a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009d90 tx=0x7f1d3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7085c082782e28e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:22:16.153 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d37fff700 1 -- 172.21.15.22:0/1373915377 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c00e070 con 0x7f1d440cc800 2023-07-06T01:22:16.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.145+0000 7f1d37fff700 1 -- 172.21.15.22:0/1373915377 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3c00bb90 con 0x7f1d440cc800 2023-07-06T01:22:16.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.146+0000 7f1d37fff700 1 -- 172.21.15.22:0/1373915377 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c003a20 con 0x7f1d440cc800 2023-07-06T01:22:16.154 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.146+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1373915377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d440cc800 msgr2=0x7f1d4406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.146+0000 7f1d4b2c5700 1 --2- 172.21.15.22:0/1373915377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d440cc800 0x7f1d4406d2a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009d90 tx=0x7f1d3c00b330 comp rx=0 tx=0).stop 2023-07-06T01:22:16.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.146+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1373915377 shutdown_connections 2023-07-06T01:22:16.155 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.146+0000 7f1d4b2c5700 1 --2- 172.21.15.22:0/1373915377 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d440cc800 0x7f1d4406d2a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.146+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1373915377 >> 172.21.15.22:0/1373915377 conn(0x7f1d440b17b0 msgr2=0x7f1d440b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:22:16.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.146+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1373915377 shutdown_connections 2023-07-06T01:22:16.156 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.146+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1373915377 wait complete. 2023-07-06T01:22:16.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.148+0000 7f1d4b2c5700 1 Processor -- start 2023-07-06T01:22:16.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.148+0000 7f1d4b2c5700 1 -- start start 2023-07-06T01:22:16.159 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d4b2c5700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d440cc800 0x7f1d440b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d4b2c5700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d441610e0 0x7f1d440c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d4b2c5700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d441614b0 0x7f1d441f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d4b2c5700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440b0fc0 con 0x7f1d441610e0 2023-07-06T01:22:16.160 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d4b2c5700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440b03b0 con 0x7f1d441614b0 2023-07-06T01:22:16.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d4b2c5700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440b0690 con 0x7f1d440cc800 2023-07-06T01:22:16.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d48860700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d441610e0 0x7f1d440c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d48860700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d441610e0 0x7f1d440c38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48456/0 (socket says 172.21.15.22:48456) 2023-07-06T01:22:16.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d48860700 1 -- 172.21.15.22:0/1230859487 learned_addr learned my addr 172.21.15.22:0/1230859487 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:22:16.161 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d49862700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d441614b0 0x7f1d441f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d49061700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d440cc800 0x7f1d440b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d48860700 1 -- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d440cc800 msgr2=0x7f1d440b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d48860700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d440cc800 0x7f1d440b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d48860700 1 -- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d441614b0 msgr2=0x7f1d441f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d48860700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d441614b0 0x7f1d441f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.162 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d48860700 1 -- 172.21.15.22:0/1230859487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c009a70 con 0x7f1d441610e0 2023-07-06T01:22:16.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d49862700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d441614b0 0x7f1d441f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:22:16.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.149+0000 7f1d49061700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d440cc800 0x7f1d440b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:22:16.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.150+0000 7f1d48860700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d441610e0 0x7f1d440c38b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800c4a0 tx=0x7f1d3800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:22:16.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.150+0000 7f1d35ffb700 1 -- 172.21.15.22:0/1230859487 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38011790 con 0x7f1d441610e0 2023-07-06T01:22:16.163 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.150+0000 7f1d35ffb700 1 -- 172.21.15.22:0/1230859487 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d380118f0 con 0x7f1d441610e0 2023-07-06T01:22:16.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.150+0000 7f1d35ffb700 1 -- 172.21.15.22:0/1230859487 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38011ca0 con 0x7f1d441610e0 2023-07-06T01:22:16.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.150+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1230859487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d440b0970 con 0x7f1d441610e0 2023-07-06T01:22:16.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.150+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1230859487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d440b58d0 con 0x7f1d441610e0 2023-07-06T01:22:16.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.151+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1230859487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d440c6d10 con 0x7f1d441610e0 2023-07-06T01:22:16.164 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.151+0000 7f1d35ffb700 1 -- 172.21.15.22:0/1230859487 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d3801c460 con 0x7f1d441610e0 2023-07-06T01:22:16.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.152+0000 7f1d35ffb700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d2806ace0 0x7f1d2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.152+0000 7f1d35ffb700 1 -- 172.21.15.22:0/1230859487 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1d38088fb0 con 0x7f1d441610e0 2023-07-06T01:22:16.165 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.152+0000 7f1d49061700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d2806ace0 0x7f1d2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.152+0000 7f1d49061700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d2806ace0 0x7f1d2806d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1d40000f60 tx=0x7f1d40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:22:16.166 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.156+0000 7f1d35ffb700 1 -- 172.21.15.22:0/1230859487 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1d380bb310 con 0x7f1d441610e0 2023-07-06T01:22:16.351 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.348+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1230859487 --> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d4406ca90 con 0x7f1d2806ace0 2023-07-06T01:22:16.354 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.352+0000 7f1d35ffb700 1 -- 172.21.15.22:0/1230859487 <== mgr.4103 v2:172.21.15.103:6832/1225712448 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73472 (secure 0 0 0) 0x7f1d400070d0 con 0x7f1d2806ace0 2023-07-06T01:22:16.356 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:22:16.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.354+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d2806ace0 msgr2=0x7f1d2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.358 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.354+0000 7f1d4b2c5700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d2806ace0 0x7f1d2806d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1d40000f60 tx=0x7f1d40009040 comp rx=0 tx=0).stop 2023-07-06T01:22:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.354+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1230859487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d441610e0 msgr2=0x7f1d440c38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.354+0000 7f1d4b2c5700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d441610e0 0x7f1d440c38b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800c4a0 tx=0x7f1d3800c860 comp rx=0 tx=0).stop 2023-07-06T01:22:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.354+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1230859487 shutdown_connections 2023-07-06T01:22:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.354+0000 7f1d4b2c5700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f1d2806ace0 0x7f1d2806d190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.354+0000 7f1d4b2c5700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d441614b0 0x7f1d441f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.354+0000 7f1d4b2c5700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1d440cc800 0x7f1d440b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.361 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.354+0000 7f1d4b2c5700 1 --2- 172.21.15.22:0/1230859487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d441610e0 0x7f1d440c38b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.354+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1230859487 >> 172.21.15.22:0/1230859487 conn(0x7f1d440b17b0 msgr2=0x7f1d44070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:22:16.362 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.354+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1230859487 shutdown_connections 2023-07-06T01:22:16.363 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.354+0000 7f1d4b2c5700 1 -- 172.21.15.22:0/1230859487 wait complete. 2023-07-06T01:22:16.363 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2023-07-06T01:22:16.369 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2023-07-06T01:22:16.041653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":318024,"kb_used_data":4872,"kb_used_omap":0,"kb_used_meta":313088,"kb_avail":754656696,"statfs":{"total":773094113280,"available":772768456704,"internally_reserved":0,"allocated":4988928,"data_stored":2801616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":320602112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002847"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:22:01.540161+0000","last_change":"2023-07-06T01:22:01.540210+0000","last_active":"2023-07-06T01:22:01.540161+0000","last_peered":"2023-07-06T01:22:01.540161+0000","last_clean":"2023-07-06T01:22:01.540161+0000","last_became_active":"2023-07-06T01:19:20.073886+0000","last_became_peered":"2023-07-06T01:19:20.073886+0000","last_unstale":"2023-07-06T01:22:01.540161+0000","last_undegraded":"2023-07-06T01:22:01.540161+0000","last_fullsized":"2023-07-06T01:22:01.540161+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:22:01.540034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:22:01.540034+0000","last_clean_scrub_stamp":"2023-07-06T01:22:01.540034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:58:50.215316+0000","scrub_duration":0.0035792490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:58.442575+0000","last_change":"2023-07-06T01:21:58.442626+0000","last_active":"2023-07-06T01:21:58.442575+0000","last_peered":"2023-07-06T01:21:58.442575+0000","last_clean":"2023-07-06T01:21:58.442575+0000","last_became_active":"2023-07-06T01:19:19.586710+0000","last_became_peered":"2023-07-06T01:19:19.586710+0000","last_unstale":"2023-07-06T01:21:58.442575+0000","last_undegraded":"2023-07-06T01:21:58.442575+0000","last_fullsized":"2023-07-06T01:21:58.442575+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:21:58.442456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:21:58.442456+0000","last_clean_scrub_stamp":"2023-07-06T01:21:58.442456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:58:47.117739+0000","scrub_duration":0.0042956219999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":59,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:53.139879+0000","last_change":"2023-07-06T01:21:53.139916+0000","last_active":"2023-07-06T01:21:53.139879+0000","last_peered":"2023-07-06T01:21:53.139879+0000","last_clean":"2023-07-06T01:21:53.139879+0000","last_became_active":"2023-07-06T01:19:19.583782+0000","last_became_peered":"2023-07-06T01:19:19.583782+0000","last_unstale":"2023-07-06T01:21:53.139879+0000","last_undegraded":"2023-07-06T01:21:53.139879+0000","last_fullsized":"2023-07-06T01:21:53.139879+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:21:53.139796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:21:53.139796+0000","last_clean_scrub_stamp":"2023-07-06T01:21:53.139796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:24:37.282299+0000","scrub_duration":0.0029849210000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'119","reported_seq":213,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:58.088458+0000","last_change":"2023-07-06T01:21:58.088513+0000","last_active":"2023-07-06T01:21:58.088458+0000","last_peered":"2023-07-06T01:21:58.088458+0000","last_clean":"2023-07-06T01:21:58.088458+0000","last_became_active":"2023-07-06T01:19:19.586186+0000","last_became_peered":"2023-07-06T01:19:19.586186+0000","last_unstale":"2023-07-06T01:21:58.088458+0000","last_undegraded":"2023-07-06T01:21:58.088458+0000","last_fullsized":"2023-07-06T01:21:58.088458+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'119","last_scrub_stamp":"2023-07-06T01:21:58.088362+0000","last_deep_scrub":"41'119","last_deep_scrub_stamp":"2023-07-06T01:21:58.088362+0000","last_clean_scrub_stamp":"2023-07-06T01:21:58.088362+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T10:58:46.763644+0000","scrub_duration":0.0070145260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":61,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:59.088823+0000","last_change":"2023-07-06T01:21:59.088870+0000","last_active":"2023-07-06T01:21:59.088823+0000","last_peered":"2023-07-06T01:21:59.088823+0000","last_clean":"2023-07-06T01:21:59.088823+0000","last_became_active":"2023-07-06T01:19:19.586109+0000","last_became_peered":"2023-07-06T01:19:19.586109+0000","last_unstale":"2023-07-06T01:21:59.088823+0000","last_undegraded":"2023-07-06T01:21:59.088823+0000","last_fullsized":"2023-07-06T01:21:59.088823+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:21:59.088719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:21:59.088719+0000","last_clean_scrub_stamp":"2023-07-06T01:21:59.088719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T11:15:32.540990+0000","scrub_duration":0.003364372,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":61,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:22:00.063770+0000","last_change":"2023-07-06T01:22:00.063823+0000","last_active":"2023-07-06T01:22:00.063770+0000","last_peered":"2023-07-06T01:22:00.063770+0000","last_clean":"2023-07-06T01:22:00.063770+0000","last_became_active":"2023-07-06T01:19:19.586314+0000","last_became_peered":"2023-07-06T01:19:19.586314+0000","last_unstale":"2023-07-06T01:22:00.063770+0000","last_undegraded":"2023-07-06T01:22:00.063770+0000","last_fullsized":"2023-07-06T01:22:00.063770+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:22:00.063655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:22:00.063655+0000","last_clean_scrub_stamp":"2023-07-06T01:22:00.063655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T07:16:45.325822+0000","scrub_duration":0.0023293789999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:57.490934+0000","last_change":"2023-07-06T01:21:57.490981+0000","last_active":"2023-07-06T01:21:57.490934+0000","last_peered":"2023-07-06T01:21:57.490934+0000","last_clean":"2023-07-06T01:21:57.490934+0000","last_became_active":"2023-07-06T01:19:19.583775+0000","last_became_peered":"2023-07-06T01:19:19.583775+0000","last_unstale":"2023-07-06T01:21:57.490934+0000","last_undegraded":"2023-07-06T01:21:57.490934+0000","last_fullsized":"2023-07-06T01:21:57.490934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:21:57.490808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:21:57.490808+0000","last_clean_scrub_stamp":"2023-07-06T01:21:57.490808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T06:02:07.581612+0000","scrub_duration":0.004239072,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":59,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:59.273002+0000","last_change":"2023-07-06T01:21:59.273051+0000","last_active":"2023-07-06T01:21:59.273002+0000","last_peered":"2023-07-06T01:21:59.273002+0000","last_clean":"2023-07-06T01:21:59.273002+0000","last_became_active":"2023-07-06T01:19:19.586153+0000","last_became_peered":"2023-07-06T01:19:19.586153+0000","last_unstale":"2023-07-06T01:21:59.273002+0000","last_undegraded":"2023-07-06T01:21:59.273002+0000","last_fullsized":"2023-07-06T01:21:59.273002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:21:59.272875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:21:59.272875+0000","last_clean_scrub_stamp":"2023-07-06T01:21:59.272875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T04:14:38.890316+0000","scrub_duration":0.0032265610000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":59,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-07-06T01:21:55.299794+0000","last_change":"2023-07-06T01:21:55.299844+0000","last_active":"2023-07-06T01:21:55.299794+0000","last_peered":"2023-07-06T01:21:55.299794+0000","last_clean":"2023-07-06T01:21:55.299794+0000","last_became_active":"2023-07-06T01:19:19.583373+0000","last_became_peered":"2023-07-06T01:19:19.583373+0000","last_unstale":"2023-07-06T01:21:55.299794+0000","last_undegraded":"2023-07-06T01:21:55.299794+0000","last_fullsized":"2023-07-06T01:21:55.299794+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T01:21:55.299669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T01:21:55.299669+0000","last_clean_scrub_stamp":"2023-07-06T01:21:55.299669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T03:15:54.336123+0000","scrub_duration":0.0034794800000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54236,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317604,"statfs":{"total":96636764160,"available":96581226496,"internally_reserved":0,"allocated":1069056,"data_stored":792978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46500000000000002,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.623,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.50700000000000001,"15min":0.51300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.497},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.497},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48999999999999999,"15min":0.49199999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69699999999999995,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.55500000000000005,"15min":0.56699999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.61499999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.42699999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72199999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57499999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.38},{"interface":"front","average":{"1min":0.505,"5min":0.58299999999999996,"15min":0.59599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.439}]},{"osd":5,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69499999999999995,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.68300000000000005,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.66100000000000003,"15min":0.67700000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71899999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.51200000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53656,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94318184,"statfs":{"total":96636764160,"available":96581820416,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68600000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.36799999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.38900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.68799999999999994,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.80800000000000005}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58299999999999996,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.74299999999999999,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.76100000000000001}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79400000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.746}]},{"osd":5,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.32800000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72199999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.68500000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.36199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79400000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64600000000000002,"15min":0.65800000000000003},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66600000000000004,"15min":0.67100000000000004},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81200000000000006,"5min":0.872,"15min":0.872},"last":0.79500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36188,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335652,"statfs":{"total":96636764160,"available":96599707648,"internally_reserved":0,"allocated":1069056,"data_stored":792978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":1.2589999999999999,"15min":1.4139999999999999},"min":{"1min":0.23200000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.51300000000000001,"5min":12.23,"15min":12.23},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.312,"5min":1.5329999999999999,"15min":1.7370000000000001},"min":{"1min":0.252,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.375,"5min":15.208,"15min":15.208},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.89900000000000002,"15min":0.98499999999999999},"min":{"1min":0.22500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":1.5189999999999999,"5min":12.074,"15min":12.074},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":1.306,"15min":1.458},"min":{"1min":0.26500000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73999999999999999,"5min":11.988,"15min":11.988},"last":0.60499999999999998}]},{"osd":3,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":1.8120000000000001,"15min":2.052},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.47999999999999998,"5min":18.963000000000001,"15min":18.963000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.34799999999999998,"5min":1.5800000000000001,"15min":1.7849999999999999},"min":{"1min":0.24399999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.41299999999999998,"5min":15.244999999999999,"15min":15.244999999999999},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":1.127,"15min":1.2010000000000001},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.6080000000000001,"5min":11.923,"15min":11.923},"last":0.62},{"interface":"front","average":{"1min":0.76800000000000002,"5min":1.5249999999999999,"15min":1.651},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.5950000000000002,"5min":12.019,"15min":12.019},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":1.099,"15min":1.1850000000000001},"min":{"1min":0.40100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79700000000000004,"5min":11.999000000000001,"15min":11.999000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":1.5,"15min":1.6499999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88800000000000001,"5min":11.951000000000001,"15min":11.951000000000001},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":1.788,"15min":1.9690000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.7490000000000001,"5min":15.167999999999999,"15min":15.167999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":1.1479999999999999,"15min":1.236},"min":{"1min":0.47799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93799999999999994,"5min":12.041,"15min":12.041},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":1.8180000000000001,"15min":2.008},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.625,"5min":15.115,"15min":15.115},"last":0.746},{"interface":"front","average":{"1min":0.69099999999999995,"5min":1.163,"15min":1.2410000000000001},"min":{"1min":0.45700000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8200000000000001,"5min":11.965,"15min":11.965},"last":0.75800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.35499999999999998,"15min":0.35999999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.60199999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45400000000000001,"15min":0.45500000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.66800000000000004,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.438},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0880000000000001,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.57799999999999996}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.45400000000000001,"15min":0.46400000000000002},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.88800000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.45700000000000002,"15min":0.46500000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.91400000000000003,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.41499999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.97199999999999998,"5min":2.097,"15min":2.097},"last":0.625},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.62},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.196,"5min":0.71699999999999997,"15min":0.63700000000000001},"min":{"1min":0.44,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":11.619,"5min":11.619,"15min":11.619},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":0.80600000000000005,"15min":0.73199999999999998},"min":{"1min":0.50800000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":11.603999999999999,"5min":11.603999999999999,"15min":11.603999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.93899999999999995,"15min":0.98399999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.518,"5min":6.3049999999999997,"15min":6.3049999999999997},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.94799999999999995,"15min":0.99099999999999999},"min":{"1min":0.48499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.609,"5min":6.2430000000000003,"15min":6.2430000000000003},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.2430000000000001,"5min":0.74399999999999999,"15min":0.66100000000000003},"min":{"1min":0.46200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":11.663,"5min":11.663,"15min":11.663},"last":0.76000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.25700000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.373},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.34899999999999998,"15min":0.33900000000000002},"min":{"1min":0.28799999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.371,"15min":0.36699999999999999},"min":{"1min":0.27100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.495}]},{"osd":3,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.34200000000000003,"15min":0.33200000000000002},"min":{"1min":0.28899999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.42399999999999999,"15min":0.42199999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61199999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.47899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.47099999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.629}]},{"osd":5,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jul 6 01:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.71799999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.45300000000000001,"15min":0.46999999999999997},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67800000000000005,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.46200000000000002,"15min":0.47799999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.52800000000000002,"5min":1.51,"15min":1.51},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.25700000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64800000000000002,"5min":1.331,"15min":1.331},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.53400000000000003,"15min":0.56000000000000005},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.58099999999999996,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.53700000000000003,"15min":0.56100000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53600000000000003,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.53700000000000003,"15min":0.56499999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.501,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.67500000000000004,"15min":0.69499999999999995},"min":{"1min":0.44700000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.71999999999999997,"5min":1.663,"15min":1.663},"last":0.879},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.66500000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71499999999999997,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.81399999999999995}]},{"osd":5,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.64200000000000002,"15min":0.65800000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.70199999999999996,"5min":1.667,"15min":1.667},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.68400000000000005,"15min":0.69999999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77500000000000002,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.83099999999999996}]},{"osd":6,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.65700000000000003,"15min":0.67400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78100000000000003,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.78800000000000003,"15min":0.82599999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.76500000000000001,"5min":3.8130000000000002,"15min":3.8130000000000002},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Jul 6 01:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.72399999999999998,"15min":0.753},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72599999999999998,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.75900000000000001,"15min":0.78200000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92600000000000005,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.76500000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51700000000000002,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.61899999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.497}]},{"osd":1,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.36199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63100000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.53300000000000003,"15min":0.54400000000000004},"min":{"1min":0.32500000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.55000000000000004}]},{"osd":2,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.41599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.61699999999999999,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73099999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81200000000000006,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.69399999999999995}]},{"osd":6,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.58299999999999996,"15min":0.59499999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jul 6 01:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.67100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":475136,"data_stored":202610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.48899999999999999,"15min":0.497},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64200000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.503,"15min":0.503},"min":{"1min":0.38200000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.66500000000000004}]},{"osd":1,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.54800000000000004,"15min":0.55600000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66700000000000004,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.33000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.64600000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.69899999999999995}]},{"osd":2,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51400000000000001,"15min":0.52000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72999999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67400000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.54100000000000004,"15min":0.55000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.68600000000000005,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.627},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56999999999999995,"15min":0.57699999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.66200000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54100000000000004,"15min":0.54600000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.67800000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.63200000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.48299999999999998}]},{"osd":7,"last update":"Thu Jul 6 01:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71199999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59899999999999998,"15min":0.60799999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.66000000000000003,"5min":0.87,"15min":0.87},"last":0.58599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T01:22:16.372 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-07-06T01:22:16.497 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.494+0000 7f81eb23b700 1 Processor -- start 2023-07-06T01:22:16.498 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.495+0000 7f81eb23b700 1 -- start start 2023-07-06T01:22:16.499 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.495+0000 7f81eb23b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81e415cb50 0x7f81e406a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.495+0000 7f81eb23b700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f81e406a540 0x7f81e40ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.495+0000 7f81eb23b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f81e40cf090 0x7f81e406cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.495+0000 7f81eb23b700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e40af440 con 0x7f81e40cf090 2023-07-06T01:22:16.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.495+0000 7f81eb23b700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e40ca2b0 con 0x7f81e406a540 2023-07-06T01:22:16.500 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.495+0000 7f81eb23b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e406a970 con 0x7f81e415cb50 2023-07-06T01:22:16.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.495+0000 7f81e97d8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f81e40cf090 0x7f81e406cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.496+0000 7f81e97d8700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f81e40cf090 0x7f81e406cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48466/0 (socket says 172.21.15.22:48466) 2023-07-06T01:22:16.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.496+0000 7f81e97d8700 1 -- 172.21.15.22:0/3349476668 learned_addr learned my addr 172.21.15.22:0/3349476668 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:22:16.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.496+0000 7f81e97d8700 1 -- 172.21.15.22:0/3349476668 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81e415cb50 msgr2=0x7f81e406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:22:16.501 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.496+0000 7f81e8fd7700 1 --2- 172.21.15.22:0/3349476668 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81e415cb50 0x7f81e406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.496+0000 7f81e3fff700 1 --2- 172.21.15.22:0/3349476668 >> v2:172.21.15.103:3300/0 conn(0x7f81e406a540 0x7f81e40ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.496+0000 7f81e97d8700 1 --2- 172.21.15.22:0/3349476668 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81e415cb50 0x7f81e406a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.496+0000 7f81e97d8700 1 -- 172.21.15.22:0/3349476668 >> v2:172.21.15.103:3300/0 conn(0x7f81e406a540 msgr2=0x7f81e40ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:22:16.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.496+0000 7f81e97d8700 1 --2- 172.21.15.22:0/3349476668 >> v2:172.21.15.103:3300/0 conn(0x7f81e406a540 0x7f81e40ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.502 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.496+0000 7f81e97d8700 1 -- 172.21.15.22:0/3349476668 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e40cfc90 con 0x7f81e40cf090 2023-07-06T01:22:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.496+0000 7f81e97d8700 1 --2- 172.21.15.22:0/3349476668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e40cf090 0x7f81e406cae0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f81e40d0d60 tx=0x7f81dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b741061bbeeee142 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:22:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.496+0000 7f81e37fe700 1 -- 172.21.15.22:0/3349476668 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00e070 con 0x7f81e40cf090 2023-07-06T01:22:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.496+0000 7f81e37fe700 1 -- 172.21.15.22:0/3349476668 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc00b980 con 0x7f81e40cf090 2023-07-06T01:22:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.496+0000 7f81e37fe700 1 -- 172.21.15.22:0/3349476668 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00bd30 con 0x7f81e40cf090 2023-07-06T01:22:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.497+0000 7f81eb23b700 1 -- 172.21.15.22:0/3349476668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e40cf090 msgr2=0x7f81e406cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.503 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.497+0000 7f81eb23b700 1 --2- 172.21.15.22:0/3349476668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e40cf090 0x7f81e406cae0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f81e40d0d60 tx=0x7f81dc00b330 comp rx=0 tx=0).stop 2023-07-06T01:22:16.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.497+0000 7f81eb23b700 1 -- 172.21.15.22:0/3349476668 shutdown_connections 2023-07-06T01:22:16.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.497+0000 7f81eb23b700 1 --2- 172.21.15.22:0/3349476668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e40cf090 0x7f81e406cae0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.497+0000 7f81eb23b700 1 --2- 172.21.15.22:0/3349476668 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81e415cb50 0x7f81e406a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.497+0000 7f81eb23b700 1 --2- 172.21.15.22:0/3349476668 >> v2:172.21.15.103:3300/0 conn(0x7f81e406a540 0x7f81e40ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.504 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.497+0000 7f81eb23b700 1 -- 172.21.15.22:0/3349476668 >> 172.21.15.22:0/3349476668 conn(0x7f81e406b110 msgr2=0x7f81e406b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:22:16.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.497+0000 7f81eb23b700 1 -- 172.21.15.22:0/3349476668 shutdown_connections 2023-07-06T01:22:16.505 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.497+0000 7f81eb23b700 1 -- 172.21.15.22:0/3349476668 wait complete. 2023-07-06T01:22:16.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.498+0000 7f81eb23b700 1 Processor -- start 2023-07-06T01:22:16.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.498+0000 7f81eb23b700 1 -- start start 2023-07-06T01:22:16.507 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81eb23b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e406a540 0x7f81e415d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81eb23b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81e40cf090 0x7f81e4153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81eb23b700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81e415cb50 0x7f81e40b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81eb23b700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e4153770 con 0x7f81e406a540 2023-07-06T01:22:16.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81eb23b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40b5a10 con 0x7f81e40cf090 2023-07-06T01:22:16.508 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81eb23b700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40b32a0 con 0x7f81e415cb50 2023-07-06T01:22:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81e8fd7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e406a540 0x7f81e415d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81e8fd7700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e406a540 0x7f81e415d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48468/0 (socket says 172.21.15.22:48468) 2023-07-06T01:22:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81e8fd7700 1 -- 172.21.15.22:0/1330686278 learned_addr learned my addr 172.21.15.22:0/1330686278 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:22:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81e3fff700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81e40cf090 0x7f81e4153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.509 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81e97d8700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81e415cb50 0x7f81e40b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81e8fd7700 1 -- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81e415cb50 msgr2=0x7f81e40b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81e8fd7700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81e415cb50 0x7f81e40b54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81e8fd7700 1 -- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81e40cf090 msgr2=0x7f81e4153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81e8fd7700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81e40cf090 0x7f81e4153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81e8fd7700 1 -- 172.21.15.22:0/1330686278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc009b30 con 0x7f81e406a540 2023-07-06T01:22:16.510 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81e3fff700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81e40cf090 0x7f81e4153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T01:22:16.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.499+0000 7f81e8fd7700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e406a540 0x7f81e415d930 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f81d800c7f0 tx=0x7f81d800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:22:16.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.500+0000 7f81e17fa700 1 -- 172.21.15.22:0/1330686278 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8011800 con 0x7f81e406a540 2023-07-06T01:22:16.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.500+0000 7f81e17fa700 1 -- 172.21.15.22:0/1330686278 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d8011960 con 0x7f81e406a540 2023-07-06T01:22:16.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.500+0000 7f81eb23b700 1 -- 172.21.15.22:0/1330686278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e415e030 con 0x7f81e406a540 2023-07-06T01:22:16.511 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.500+0000 7f81e17fa700 1 -- 172.21.15.22:0/1330686278 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8011cf0 con 0x7f81e406a540 2023-07-06T01:22:16.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.500+0000 7f81eb23b700 1 -- 172.21.15.22:0/1330686278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e40b2f30 con 0x7f81e406a540 2023-07-06T01:22:16.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.501+0000 7f81eb23b700 1 -- 172.21.15.22:0/1330686278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e40b2ab0 con 0x7f81e406a540 2023-07-06T01:22:16.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.501+0000 7f81e17fa700 1 -- 172.21.15.22:0/1330686278 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f81d8018360 con 0x7f81e406a540 2023-07-06T01:22:16.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.501+0000 7f81e17fa700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f81c806ace0 0x7f81c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.512 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.502+0000 7f81e17fa700 1 -- 172.21.15.22:0/1330686278 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f81d80890c0 con 0x7f81e406a540 2023-07-06T01:22:16.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.502+0000 7f81e3fff700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f81c806ace0 0x7f81c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.502+0000 7f81e3fff700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f81c806ace0 0x7f81c806d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f81e41f68f0 tx=0x7f81d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:22:16.513 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.506+0000 7f81e17fa700 1 -- 172.21.15.22:0/1330686278 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f81e40b2ab0 con 0x7f81e406a540 2023-07-06T01:22:16.705 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.702+0000 7f81eb23b700 1 -- 172.21.15.22:0/1330686278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f81e40b2ab0 con 0x7f81e406a540 2023-07-06T01:22:16.708 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.706+0000 7f81e17fa700 1 -- 172.21.15.22:0/1330686278 <== mon.0 v2:172.21.15.22:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f81d8054470 con 0x7f81e406a540 2023-07-06T01:22:16.709 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.706+0000 7f81e17fa700 1 -- 172.21.15.22:0/1330686278 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f81e40b2ab0 con 0x7f81e406a540 2023-07-06T01:22:16.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.708+0000 7f81eb23b700 1 -- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f81c806ace0 msgr2=0x7f81c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.711 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.708+0000 7f81eb23b700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f81c806ace0 0x7f81c806d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f81e41f68f0 tx=0x7f81d0008040 comp rx=0 tx=0).stop 2023-07-06T01:22:16.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.708+0000 7f81eb23b700 1 -- 172.21.15.22:0/1330686278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e406a540 msgr2=0x7f81e415d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.712 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.708+0000 7f81eb23b700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e406a540 0x7f81e415d930 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f81d800c7f0 tx=0x7f81d800cbb0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.708+0000 7f81eb23b700 1 -- 172.21.15.22:0/1330686278 shutdown_connections 2023-07-06T01:22:16.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.708+0000 7f81eb23b700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81e415cb50 0x7f81e40b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.708+0000 7f81eb23b700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81e406a540 0x7f81e415d930 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.713 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.708+0000 7f81eb23b700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7f81c806ace0 0x7f81c806d190 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.708+0000 7f81eb23b700 1 --2- 172.21.15.22:0/1330686278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81e40cf090 0x7f81e4153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.708+0000 7f81eb23b700 1 -- 172.21.15.22:0/1330686278 >> 172.21.15.22:0/1330686278 conn(0x7f81e406b110 msgr2=0x7f81e40cada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:22:16.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.709+0000 7f81eb23b700 1 -- 172.21.15.22:0/1330686278 shutdown_connections 2023-07-06T01:22:16.714 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.709+0000 7f81eb23b700 1 -- 172.21.15.22:0/1330686278 wait complete. 2023-07-06T01:22:16.723 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T01:22:16.724 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T01:22:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.846+0000 7fefbc9fe700 1 Processor -- start 2023-07-06T01:22:16.849 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.846+0000 7fefbc9fe700 1 -- start start 2023-07-06T01:22:16.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefbc9fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb815a780 0x7fefb806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefbc9fe700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fefb815f100 0x7fefb806a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.851 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefbc9fe700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fefb80cfbf0 0x7fefb80d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefbc9fe700 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb815e3d0 con 0x7fefb815f100 2023-07-06T01:22:16.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefbc9fe700 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb815ec70 con 0x7fefb80cfbf0 2023-07-06T01:22:16.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefbc9fe700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80d0920 con 0x7fefb815a780 2023-07-06T01:22:16.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb5d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fefb815f100 0x7fefb806a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.852 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb5d9b700 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fefb815f100 0x7fefb806a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48478/0 (socket says 172.21.15.22:48478) 2023-07-06T01:22:16.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb5d9b700 1 -- 172.21.15.22:0/2939588083 learned_addr learned my addr 172.21.15.22:0/2939588083 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:22:16.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb659c700 1 --2- 172.21.15.22:0/2939588083 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb815a780 0x7fefb806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb5d9b700 1 -- 172.21.15.22:0/2939588083 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb815a780 msgr2=0x7fefb806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:22:16.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb5d9b700 1 --2- 172.21.15.22:0/2939588083 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb815a780 0x7fefb806a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.853 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb5d9b700 1 -- 172.21.15.22:0/2939588083 >> v2:172.21.15.103:3300/0 conn(0x7fefb80cfbf0 msgr2=0x7fefb80d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T01:22:16.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb6d9d700 1 --2- 172.21.15.22:0/2939588083 >> v2:172.21.15.103:3300/0 conn(0x7fefb80cfbf0 0x7fefb80d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb5d9b700 1 --2- 172.21.15.22:0/2939588083 >> v2:172.21.15.103:3300/0 conn(0x7fefb80cfbf0 0x7fefb80d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.854 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb5d9b700 1 -- 172.21.15.22:0/2939588083 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb815e620 con 0x7fefb815f100 2023-07-06T01:22:16.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb659c700 1 --2- 172.21.15.22:0/2939588083 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb815a780 0x7fefb806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:22:16.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb6d9d700 1 --2- 172.21.15.22:0/2939588083 >> v2:172.21.15.103:3300/0 conn(0x7fefb80cfbf0 0x7fefb80d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:22:16.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb5d9b700 1 --2- 172.21.15.22:0/2939588083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb815f100 0x7fefb806a540 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fefb80cc800 tx=0x7fefa0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4a246e2816e964 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:22:16.855 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.847+0000 7fefb559a700 1 -- 172.21.15.22:0/2939588083 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa000b870 con 0x7fefb815f100 2023-07-06T01:22:16.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.848+0000 7fefb559a700 1 -- 172.21.15.22:0/2939588083 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fefa000b9d0 con 0x7fefb815f100 2023-07-06T01:22:16.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.848+0000 7fefb559a700 1 -- 172.21.15.22:0/2939588083 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa000bd80 con 0x7fefb815f100 2023-07-06T01:22:16.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.848+0000 7fefbc9fe700 1 -- 172.21.15.22:0/2939588083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb815f100 msgr2=0x7fefb806a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.856 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.848+0000 7fefbc9fe700 1 --2- 172.21.15.22:0/2939588083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb815f100 0x7fefb806a540 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fefb80cc800 tx=0x7fefa0009bd0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.848+0000 7fefbc9fe700 1 -- 172.21.15.22:0/2939588083 shutdown_connections 2023-07-06T01:22:16.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.848+0000 7fefbc9fe700 1 --2- 172.21.15.22:0/2939588083 >> v2:172.21.15.103:3300/0 conn(0x7fefb80cfbf0 0x7fefb80d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.857 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.848+0000 7fefbc9fe700 1 --2- 172.21.15.22:0/2939588083 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb815a780 0x7fefb806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.848+0000 7fefbc9fe700 1 --2- 172.21.15.22:0/2939588083 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb815f100 0x7fefb806a540 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.848+0000 7fefbc9fe700 1 -- 172.21.15.22:0/2939588083 >> 172.21.15.22:0/2939588083 conn(0x7fefb806b110 msgr2=0x7fefb806b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:22:16.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.848+0000 7fefbc9fe700 1 -- 172.21.15.22:0/2939588083 shutdown_connections 2023-07-06T01:22:16.858 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.848+0000 7fefbc9fe700 1 -- 172.21.15.22:0/2939588083 wait complete. 2023-07-06T01:22:16.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.849+0000 7fefbc9fe700 1 Processor -- start 2023-07-06T01:22:16.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.850+0000 7fefbc9fe700 1 -- start start 2023-07-06T01:22:16.861 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.850+0000 7fefbc9fe700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb80cfbf0 0x7fefb81f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.850+0000 7fefbc9fe700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb815a780 0x7fefb80b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.850+0000 7fefbc9fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb815f100 0x7fefb8152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.850+0000 7fefbc9fe700 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80b39d0 con 0x7fefb815a780 2023-07-06T01:22:16.862 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.850+0000 7fefbc9fe700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80b2d50 con 0x7fefb815f100 2023-07-06T01:22:16.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.850+0000 7fefbc9fe700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80b3030 con 0x7fefb80cfbf0 2023-07-06T01:22:16.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.850+0000 7fefb5d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb815a780 0x7fefb80b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.850+0000 7fefb5d9b700 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb815a780 0x7fefb80b1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48484/0 (socket says 172.21.15.22:48484) 2023-07-06T01:22:16.863 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.850+0000 7fefb5d9b700 1 -- 172.21.15.22:0/253694783 learned_addr learned my addr 172.21.15.22:0/253694783 (peer_addr_for_me v2:172.21.15.22:0/0) 2023-07-06T01:22:16.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefb6d9d700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb815f100 0x7fefb8152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefb659c700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb80cfbf0 0x7fefb81f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.864 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefb5d9b700 1 -- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb80cfbf0 msgr2=0x7fefb81f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefb5d9b700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb80cfbf0 0x7fefb81f3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefb5d9b700 1 -- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb815f100 msgr2=0x7fefb8152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:16.865 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefb5d9b700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb815f100 0x7fefb8152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:16.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefb5d9b700 1 -- 172.21.15.22:0/253694783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa0009550 con 0x7fefb815a780 2023-07-06T01:22:16.866 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefb659c700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb80cfbf0 0x7fefb81f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:22:16.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefb6d9d700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb815f100 0x7fefb8152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T01:22:16.867 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefb5d9b700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb815a780 0x7fefb80b1570 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fefa0014040 tx=0x7fefa0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:22:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefaeffd700 1 -- 172.21.15.22:0/253694783 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0004520 con 0x7fefb815a780 2023-07-06T01:22:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefaeffd700 1 -- 172.21.15.22:0/253694783 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fefa001e070 con 0x7fefb815a780 2023-07-06T01:22:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefbc9fe700 1 -- 172.21.15.22:0/253694783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb80b3310 con 0x7fefb815a780 2023-07-06T01:22:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefaeffd700 1 -- 172.21.15.22:0/253694783 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0003900 con 0x7fefb815a780 2023-07-06T01:22:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.851+0000 7fefbc9fe700 1 -- 172.21.15.22:0/253694783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb81f3c40 con 0x7fefb815a780 2023-07-06T01:22:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.852+0000 7fefbc9fe700 1 -- 172.21.15.22:0/253694783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb80b3b10 con 0x7fefb815a780 2023-07-06T01:22:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.853+0000 7fefaeffd700 1 -- 172.21.15.22:0/253694783 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fefa0003a60 con 0x7fefb815a780 2023-07-06T01:22:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.853+0000 7fefaeffd700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef9806ace0 0x7fef9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T01:22:16.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.853+0000 7fefaeffd700 1 -- 172.21.15.22:0/253694783 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fefa0020070 con 0x7fefb815a780 2023-07-06T01:22:16.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.853+0000 7fefb659c700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef9806ace0 0x7fef9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T01:22:16.870 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.854+0000 7fefb659c700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef9806ace0 0x7fef9806d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fefa8008940 tx=0x7fefa8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T01:22:16.871 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:16.857+0000 7fefaeffd700 1 -- 172.21.15.22:0/253694783 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fefb80b3b10 con 0x7fefb815a780 2023-07-06T01:22:17.103 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.101+0000 7fefbc9fe700 1 -- 172.21.15.22:0/253694783 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefb80b3b10 con 0x7fefb815a780 2023-07-06T01:22:17.104 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.101+0000 7fefaeffd700 1 -- 172.21.15.22:0/253694783 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fefa0015de0 con 0x7fefb815a780 2023-07-06T01:22:17.104 INFO:teuthology.orchestra.run.smithi022.stdout: 2023-07-06T01:22:17.105 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4766,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6836/219695916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":219695916},{"type":"v1","addr":"172.21.15.103:6836","nonce":219695916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3102932229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6835","nonce":3102932229},{"type":"v1","addr":"172.21.15.103:6837","nonce":3102932229}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3787686782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3787686782},{"type":"v1","addr":"172.21.15.22:6834","nonce":3787686782}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5508,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/361417896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":361417896},{"type":"v1","addr":"172.21.15.22:6835","nonce":361417896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-07-06T01:22:17.106 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.103+0000 7fefbc9fe700 1 -- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef9806ace0 msgr2=0x7fef9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.103+0000 7fefbc9fe700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef9806ace0 0x7fef9806d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fefa8008940 tx=0x7fefa8008040 comp rx=0 tx=0).stop 2023-07-06T01:22:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.103+0000 7fefbc9fe700 1 -- 172.21.15.22:0/253694783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb815a780 msgr2=0x7fefb80b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T01:22:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.103+0000 7fefbc9fe700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb815a780 0x7fefb80b1570 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fefa0014040 tx=0x7fefa0004750 comp rx=0 tx=0).stop 2023-07-06T01:22:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.103+0000 7fefbc9fe700 1 -- 172.21.15.22:0/253694783 shutdown_connections 2023-07-06T01:22:17.107 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.103+0000 7fefbc9fe700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:6832/1225712448,v1:172.21.15.103:6833/1225712448] conn(0x7fef9806ace0 0x7fef9806d190 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.103+0000 7fefbc9fe700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb815f100 0x7fefb8152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.103+0000 7fefbc9fe700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb80cfbf0 0x7fefb81f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.103+0000 7fefbc9fe700 1 --2- 172.21.15.22:0/253694783 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefb815a780 0x7fefb80b1570 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T01:22:17.108 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.103+0000 7fefbc9fe700 1 -- 172.21.15.22:0/253694783 >> 172.21.15.22:0/253694783 conn(0x7fefb806b110 msgr2=0x7fefb8159580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T01:22:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.103+0000 7fefbc9fe700 1 -- 172.21.15.22:0/253694783 shutdown_connections 2023-07-06T01:22:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:2023-07-06T01:22:17.103+0000 7fefbc9fe700 1 -- 172.21.15.22:0/253694783 wait complete. 2023-07-06T01:22:17.109 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2023-07-06T01:22:17.118 DEBUG:tasks.cephfs.filesystem:already dead... 2023-07-06T01:22:17.119 INFO:teuthology.misc:Shutting down mds daemons... 2023-07-06T01:22:17.119 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T01:22:17.120 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.128 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T01:22:17.129 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T01:22:17.129 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.177 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T01:22:17.178 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T01:22:17.178 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.189 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T01:22:17.190 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T01:22:17.190 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.241 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T01:22:17.242 INFO:teuthology.misc:Shutting down osd daemons... 2023-07-06T01:22:17.242 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-07-06T01:22:17.243 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.275 INFO:tasks.ceph.osd.0:Stopped 2023-07-06T01:22:17.276 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-07-06T01:22:17.276 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.343 INFO:tasks.ceph.osd.1:Stopped 2023-07-06T01:22:17.344 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-07-06T01:22:17.344 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.422 INFO:tasks.ceph.osd.2:Stopped 2023-07-06T01:22:17.423 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-07-06T01:22:17.424 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.498 INFO:tasks.ceph.osd.3:Stopped 2023-07-06T01:22:17.499 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-07-06T01:22:17.499 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.529 INFO:tasks.ceph.osd.4:Stopped 2023-07-06T01:22:17.529 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-07-06T01:22:17.530 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.602 INFO:tasks.ceph.osd.5:Stopped 2023-07-06T01:22:17.603 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-07-06T01:22:17.603 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.682 INFO:tasks.ceph.osd.6:Stopped 2023-07-06T01:22:17.682 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-07-06T01:22:17.683 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.761 INFO:tasks.ceph.osd.7:Stopped 2023-07-06T01:22:17.762 INFO:teuthology.misc:Shutting down mgr daemons... 2023-07-06T01:22:17.763 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-07-06T01:22:17.764 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.785 INFO:tasks.ceph.mgr.y:Stopped 2023-07-06T01:22:17.786 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-07-06T01:22:17.787 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.829 INFO:tasks.ceph.mgr.x:Stopped 2023-07-06T01:22:17.830 INFO:teuthology.misc:Shutting down mon daemons... 2023-07-06T01:22:17.830 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-07-06T01:22:17.831 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.842 INFO:tasks.ceph.mon.a:Stopped 2023-07-06T01:22:17.843 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-07-06T01:22:17.843 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.882 INFO:tasks.ceph.mon.b:Stopped 2023-07-06T01:22:17.882 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-07-06T01:22:17.883 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T01:22:17.934 INFO:tasks.ceph.mon.c:Stopped 2023-07-06T01:22:17.935 INFO:tasks.ceph:Checking cluster log for badness... 2023-07-06T01:22:17.935 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2023-07-06T01:22:17.970 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:22:17.971 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-07-06T01:22:18.235 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:22:18.235 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-07-06T01:22:18.330 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:22:18.331 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-07-06T01:22:18.413 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:22:18.413 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-07-06T01:22:18.497 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi103.front.sepia.ceph.com 2023-07-06T01:22:18.497 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-07-06T01:22:18.666 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi103.front.sepia.ceph.com 2023-07-06T01:22:18.667 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-07-06T01:22:18.747 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi103.front.sepia.ceph.com 2023-07-06T01:22:18.748 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-07-06T01:22:18.829 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi103.front.sepia.ceph.com 2023-07-06T01:22:18.830 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-07-06T01:22:18.904 INFO:tasks.ceph:Archiving mon data... 2023-07-06T01:22:18.906 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789/data/mon.a.tgz 2023-07-06T01:22:18.906 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2023-07-06T01:22:18.925 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.icsdQ3OOu5 2023-07-06T01:22:18.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.icsdQ3OOu5 2023-07-06T01:22:19.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.icsdQ3OOu5 2023-07-06T01:22:19.152 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.icsdQ3OOu5 is 314KB 2023-07-06T01:22:19.218 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.icsdQ3OOu5 2023-07-06T01:22:19.234 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789/data/mon.b.tgz 2023-07-06T01:22:19.235 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2023-07-06T01:22:19.254 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.xDACXDHHVV 2023-07-06T01:22:19.254 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.xDACXDHHVV 2023-07-06T01:22:19.415 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.xDACXDHHVV 2023-07-06T01:22:19.499 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.xDACXDHHVV is 417KB 2023-07-06T01:22:19.593 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.xDACXDHHVV 2023-07-06T01:22:19.609 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789/data/mon.c.tgz 2023-07-06T01:22:19.610 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2023-07-06T01:22:19.666 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.E8V3dKZELp 2023-07-06T01:22:19.667 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.E8V3dKZELp 2023-07-06T01:22:19.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.E8V3dKZELp 2023-07-06T01:22:19.891 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.E8V3dKZELp is 416KB 2023-07-06T01:22:20.058 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.E8V3dKZELp 2023-07-06T01:22:20.074 INFO:tasks.ceph:Cleaning ceph cluster... 2023-07-06T01:22:20.075 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-06T01:22:20.079 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-06T01:22:20.140 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-07-06T01:22:20.141 DEBUG:teuthology.orchestra.run.smithi022:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-06T01:22:20.151 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-06T01:22:20.186 INFO:teuthology.orchestra.run.smithi022.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-06T01:22:20.215 INFO:teuthology.orchestra.run.smithi103.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-06T01:22:20.221 INFO:tasks.ceph:Archiving crash dumps... 2023-07-06T01:22:20.223 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789/remote/smithi022/crash 2023-07-06T01:22:20.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-06T01:22:20.263 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789/remote/smithi103/crash 2023-07-06T01:22:20.264 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-06T01:22:20.291 INFO:tasks.ceph:Shutting down logrotate 2023-07-06T01:22:20.292 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:22:20.309 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T01:22:20.386 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-06T01:22:20.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-06T01:22:20.446 INFO:tasks.ceph:Compressing logs... 2023-07-06T01:22:20.447 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-06T01:22:20.460 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-06T01:22:20.488 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106734.log 2023-07-06T01:22:20.488 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122347.log 2023-07-06T01:22:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.106734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113634.log 2023-07-06T01:22:20.493 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.122347.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106734.log.gz 2023-07-06T01:22:20.494 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111549.log 2023-07-06T01:22:20.494 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123118.log 2023-07-06T01:22:20.494 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111549.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113634.log.gz 2023-07-06T01:22:20.494 INFO:teuthology.orchestra.run.smithi022.stderr: 90.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.122347.log.gz 2023-07-06T01:22:20.494 INFO:teuthology.orchestra.run.smithi022.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.113220.log 2023-07-06T01:22:20.495 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111549.log.gz 2023-07-06T01:22:20.495 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111687.log 2023-07-06T01:22:20.495 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.123118.log.gz 2023-07-06T01:22:20.495 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108526.log 2023-07-06T01:22:20.495 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113220.log.gz 2023-07-06T01:22:20.496 INFO:teuthology.orchestra.run.smithi022.stderr:gzip /var/log/ceph/ceph-client.admin.111687.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.108161.log 2023-07-06T01:22:20.496 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108526.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.111687.log.gz-5 2023-07-06T01:22:20.496 INFO:teuthology.orchestra.run.smithi022.stderr:--verbose -- /var/log/ceph/ceph-client.admin.114462.log 2023-07-06T01:22:20.496 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108526.log.gz 2023-07-06T01:22:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122277.log 2023-07-06T01:22:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107764.log 2023-07-06T01:22:20.497 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108161.log.gz 2023-07-06T01:22:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111917.log 2023-07-06T01:22:20.497 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.122277.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.114462.log.gz 2023-07-06T01:22:20.498 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109151.log 2023-07-06T01:22:20.498 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107764.log.gz 2023-07-06T01:22:20.498 INFO:teuthology.orchestra.run.smithi022.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.122277.log.gz 2023-07-06T01:22:20.498 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-07-06T01:22:20.498 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111917.log: /var/log/ceph/ceph-client.admin.109151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106378.log 2023-07-06T01:22:20.499 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111917.log.gz 2023-07-06T01:22:20.499 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114784.log 2023-07-06T01:22:20.499 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109151.log.gz 2023-07-06T01:22:20.499 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.106378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122888.log 2023-07-06T01:22:20.499 INFO:teuthology.orchestra.run.smithi022.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106378.log.gz 2023-07-06T01:22:20.500 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.115950.log 2023-07-06T01:22:20.515 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.107522.log 2023-07-06T01:22:20.515 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-07-06T01:22:20.516 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.1.107522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101029.log 2023-07-06T01:22:20.516 INFO:teuthology.orchestra.run.smithi103.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.107522.log.gz 2023-07-06T01:22:20.516 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-06T01:22:20.516 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.101029.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-07-06T01:22:20.517 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101029.log.gz 2023-07-06T01:22:20.517 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101309.log 2023-07-06T01:22:20.517 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101166.log 2023-07-06T01:22:20.517 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.108141.log 2023-07-06T01:22:20.517 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.108137.log 2023-07-06T01:22:20.518 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.101309.log: /var/log/ceph/ceph-client.admin.101166.log: /var/log/ceph/ceph-client.3.108141.log: 91.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-07-06T01:22:20.518 INFO:teuthology.orchestra.run.smithi103.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.108141.log.gz 2023-07-06T01:22:20.518 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.3.108137.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101309.log.gz 2023-07-06T01:22:20.518 INFO:teuthology.orchestra.run.smithi103.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101166.log.gz 2023-07-06T01:22:20.518 INFO:teuthology.orchestra.run.smithi103.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.108137.log.gz 2023-07-06T01:22:20.519 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-07-06T01:22:20.519 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-06T01:22:20.521 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101258.log 2023-07-06T01:22:20.522 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-07-06T01:22:20.522 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.101258.log: /var/log/ceph/ceph-osd.4.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101258.log.gz 2023-07-06T01:22:20.522 INFO:teuthology.orchestra.run.smithi103.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-06T01:22:20.522 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101121.log 2023-07-06T01:22:20.522 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-07-06T01:22:20.523 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.101121.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101121.log.gz 2023-07-06T01:22:20.528 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101075.log 2023-07-06T01:22:20.528 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101212.log 2023-07-06T01:22:20.529 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.101075.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101075.log.gz 2023-07-06T01:22:20.529 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.101212.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101212.log.gz 2023-07-06T01:22:20.529 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-07-06T01:22:20.529 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.107530.log 2023-07-06T01:22:20.529 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.1.107530.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.107530.log.gz 2023-07-06T01:22:20.529 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-07-06T01:22:20.531 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112799.log /var/log/ceph/ceph-client.admin.122888.log: 2023-07-06T01:22:20.532 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114784.log.gz 2023-07-06T01:22:20.532 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.115950.log: 89.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122888.log.gz 2023-07-06T01:22:20.532 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.0.115950.log.gz 2023-07-06T01:22:20.532 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111595.log 2023-07-06T01:22:20.532 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114324.log 2023-07-06T01:22:20.533 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.112799.log: /var/log/ceph/ceph-client.admin.111595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108921.log 2023-07-06T01:22:20.533 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112799.log.gz 2023-07-06T01:22:20.533 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1%/var/log/ceph/ceph-client.admin.114324.log: -- replaced with /var/log/ceph/ceph-client.admin.111595.log.gz 2023-07-06T01:22:20.533 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-06T01:22:20.533 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114324.log.gz 2023-07-06T01:22:20.534 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107336.log 2023-07-06T01:22:20.534 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108921.log.gz 2023-07-06T01:22:20.534 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122980.log 2023-07-06T01:22:20.534 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.107336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115212.log 2023-07-06T01:22:20.534 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.122980.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107336.log.gz 2023-07-06T01:22:20.535 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108530.log 2023-07-06T01:22:20.535 INFO:teuthology.orchestra.run.smithi022.stderr: 84.4% -- replaced with /var/log/ceph/ceph.log.gz 87.7% 2023-07-06T01:22:20.535 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.122980.log.gz 2023-07-06T01:22:20.535 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124507.log 2023-07-06T01:22:20.535 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124139.log 2023-07-06T01:22:20.536 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108530.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115212.log.gz 2023-07-06T01:22:20.536 INFO:teuthology.orchestra.run.smithi022.stderr:gzip /var/log/ceph/ceph-client.admin.124507.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.101098.log 2023-07-06T01:22:20.536 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108530.log.gz 2023-07-06T01:22:20.536 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124139.log: gzip -5 --verbose -- 87.4%/var/log/ceph/ceph-client.admin.107428.log 2023-07-06T01:22:20.537 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.124507.log.gz 2023-07-06T01:22:20.537 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.124139.log.gz 2023-07-06T01:22:20.537 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.101098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124231.log 2023-07-06T01:22:20.537 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101098.log.gz 2023-07-06T01:22:20.538 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111733.log 2023-07-06T01:22:20.538 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107428.log: /var/log/ceph/ceph-client.admin.124231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101228.log 2023-07-06T01:22:20.538 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107428.log.gz 2023-07-06T01:22:20.538 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111733.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124231.log.gz 2023-07-06T01:22:20.538 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108737.log 2023-07-06T01:22:20.538 INFO:teuthology.orchestra.run.smithi022.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111733.log.gz 2023-07-06T01:22:20.539 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.101228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106426.log 2023-07-06T01:22:20.539 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101228.log.gz 2023-07-06T01:22:20.539 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116060.log 2023-07-06T01:22:20.539 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108737.log: /var/log/ceph/ceph-client.admin.106426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115601.log 2023-07-06T01:22:20.539 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.108737.log.gz 2023-07-06T01:22:20.540 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106426.log.gz 2023-07-06T01:22:20.540 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.116060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123643.log 2023-07-06T01:22:20.540 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116060.log.gz 2023-07-06T01:22:20.540 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112228.log 2023-07-06T01:22:20.540 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111318.log 2023-07-06T01:22:20.541 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115601.log.gz 2023-07-06T01:22:20.541 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.112228.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123643.log.gz 2023-07-06T01:22:20.541 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115396.log 2023-07-06T01:22:20.541 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112228.log.gz 2023-07-06T01:22:20.541 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111318.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-07-06T01:22:20.542 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111318.log.gz 2023-07-06T01:22:20.542 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113266.log 2023-07-06T01:22:20.542 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115396.log: /var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115509.log 2023-07-06T01:22:20.542 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.115396.log.gz 2023-07-06T01:22:20.542 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107012.log 2023-07-06T01:22:20.542 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112548.log 2023-07-06T01:22:20.543 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.113266.log.gz 2023-07-06T01:22:20.543 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115509.log.gz 2023-07-06T01:22:20.543 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124415.log 2023-07-06T01:22:20.543 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106516.log 2023-07-06T01:22:20.543 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.112548.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107012.log.gz 2023-07-06T01:22:20.544 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124415.log: gzip 88.4%-5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.112548.log.gz 2023-07-06T01:22:20.544 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111042.log 2023-07-06T01:22:20.544 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.106516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123210.log 2023-07-06T01:22:20.544 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.124415.log.gz 2023-07-06T01:22:20.544 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111042.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.106516.log.gz 2023-07-06T01:22:20.545 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124645.log 2023-07-06T01:22:20.545 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111042.log.gz 2023-07-06T01:22:20.545 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114416.log 2023-07-06T01:22:20.545 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-07-06T01:22:20.545 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124645.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.123210.log.gz 2023-07-06T01:22:20.546 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107244.log 2023-07-06T01:22:20.546 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124645.log.gz 2023-07-06T01:22:20.546 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mds.a.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114416.log.gz 2023-07-06T01:22:20.546 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114878.log 2023-07-06T01:22:20.546 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123909.log 2023-07-06T01:22:20.547 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113086.log 2023-07-06T01:22:20.547 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107244.log.gz 2023-07-06T01:22:20.547 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114878.log.gz 2023-07-06T01:22:20.547 INFO:teuthology.orchestra.run.smithi022.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-07-06T01:22:20.547 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107198.log 2023-07-06T01:22:20.547 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123909.log.gz 2023-07-06T01:22:20.548 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112366.log 2023-07-06T01:22:20.548 INFO:teuthology.orchestra.run.smithi022.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113086.log.gz 2023-07-06T01:22:20.548 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107060.log 2023-07-06T01:22:20.548 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107198.log: /var/log/ceph/ceph-client.admin.112366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124001.log 2023-07-06T01:22:20.548 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107198.log.gz 2023-07-06T01:22:20.549 INFO:teuthology.orchestra.run.smithi022.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112366.log.gz 2023-07-06T01:22:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107771.log 2023-07-06T01:22:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112414.log 2023-07-06T01:22:20.549 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107060.log.gz 2023-07-06T01:22:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124001.log: /var/log/ceph/ceph-client.admin.107771.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124001.log.gz 2023-07-06T01:22:20.550 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107771.log.gz 2023-07-06T01:22:20.550 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.112414.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112414.log.gz 2023-07-06T01:22:20.550 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111226.log 2023-07-06T01:22:20.550 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107106.log 2023-07-06T01:22:20.550 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114554.log 2023-07-06T01:22:20.551 INFO:teuthology.orchestra.run.smithi022.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.111226.log.gz 2023-07-06T01:22:20.551 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109289.log 2023-07-06T01:22:20.551 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123164.log 2023-07-06T01:22:20.551 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114554.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107106.log.gz 2023-07-06T01:22:20.551 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.109289.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-07-06T01:22:20.552 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.114554.log.gz 2023-07-06T01:22:20.552 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.109289.log.gz 2023-07-06T01:22:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106872.log 2023-07-06T01:22:20.552 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.123164.log.gz 2023-07-06T01:22:20.552 INFO:teuthology.orchestra.run.smithi022.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.109059.log 2023-07-06T01:22:20.553 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.106872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115647.log 2023-07-06T01:22:20.553 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106872.log.gz 2023-07-06T01:22:20.553 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.109059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108783.log 2023-07-06T01:22:20.553 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109059.log.gz/var/log/ceph/ceph-client.admin.115647.log: 2023-07-06T01:22:20.553 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107768.log 2023-07-06T01:22:20.554 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108875.log 2023-07-06T01:22:20.554 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% 91.2% -- replaced with /var/log/ceph/ceph-client.admin.108783.log.gz 2023-07-06T01:22:20.554 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.115647.log.gz 2023-07-06T01:22:20.554 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110385.log 2023-07-06T01:22:20.555 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113864.log 2023-07-06T01:22:20.556 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107768.log.gz 2023-07-06T01:22:20.556 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108875.log.gz 2023-07-06T01:22:20.556 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.110385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111180.log 2023-07-06T01:22:20.556 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113864.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110385.log.gz 2023-07-06T01:22:20.556 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122597.log 2023-07-06T01:22:20.557 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113864.log.gz 2023-07-06T01:22:20.557 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111180.log.gz 2023-07-06T01:22:20.557 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107773.log 2023-07-06T01:22:20.557 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112902.log 2023-07-06T01:22:20.557 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.122597.log: /var/log/ceph/ceph-client.admin.107773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123551.log 2023-07-06T01:22:20.557 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122597.log.gz 2023-07-06T01:22:20.558 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107773.log.gz 2023-07-06T01:22:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.112902.log: /var/log/ceph/ceph-client.admin.123551.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112902.log.gz 2023-07-06T01:22:20.558 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123551.log.gz 2023-07-06T01:22:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124047.log 2023-07-06T01:22:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122689.log 2023-07-06T01:22:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108160.log 2023-07-06T01:22:20.559 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.124047.log.gz 2023-07-06T01:22:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.122689.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122689.log.gz 2023-07-06T01:22:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108160.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108160.log.gz 2023-07-06T01:22:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124783.log 2023-07-06T01:22:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123256.log 2023-07-06T01:22:20.560 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124783.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-07-06T01:22:20.561 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124783.log.gz 2023-07-06T01:22:20.561 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123256.log: /var/log/ceph/ceph-mds.c.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123256.log.gz 2023-07-06T01:22:20.561 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109013.log 2023-07-06T01:22:20.561 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122643.log 2023-07-06T01:22:20.561 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.109013.log: /var/log/ceph/ceph-client.admin.122643.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109013.log.gz 2023-07-06T01:22:20.562 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122643.log.gz 2023-07-06T01:22:20.562 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124553.log 2023-07-06T01:22:20.562 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109381.log 2023-07-06T01:22:20.562 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123505.log 2023-07-06T01:22:20.562 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4%/var/log/ceph/ceph-client.admin.109381.log: -- replaced with /var/log/ceph/ceph-client.admin.124553.log.gz 2023-07-06T01:22:20.563 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123505.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109381.log.gz 2023-07-06T01:22:20.563 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123505.log.gz 2023-07-06T01:22:20.563 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112994.log 2023-07-06T01:22:20.563 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114002.log 2023-07-06T01:22:20.563 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.112994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.110163.log 2023-07-06T01:22:20.564 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112994.log.gz 2023-07-06T01:22:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114692.log 2023-07-06T01:22:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.110163.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114002.log.gz 2023-07-06T01:22:20.564 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.110163.log.gz 2023-07-06T01:22:20.564 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114692.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.114692.log.gz 2023-07-06T01:22:20.565 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108369.log 2023-07-06T01:22:20.565 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108525.log 2023-07-06T01:22:20.565 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115304.log 2023-07-06T01:22:20.565 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108369.log.gz 2023-07-06T01:22:20.565 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108525.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108525.log.gz 2023-07-06T01:22:20.565 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115304.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115304.log.gz 2023-07-06T01:22:20.566 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111457.log 2023-07-06T01:22:20.566 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111088.log 2023-07-06T01:22:20.566 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111457.log: /var/log/ceph/ceph-client.admin.111088.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111457.log.gz 2023-07-06T01:22:20.566 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111088.log.gz 2023-07-06T01:22:20.566 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112460.log 2023-07-06T01:22:20.567 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107474.log 2023-07-06T01:22:20.567 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.112460.log: /var/log/ceph/ceph-client.admin.107474.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112460.log.gz 2023-07-06T01:22:20.567 INFO:teuthology.orchestra.run.smithi022.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107474.log.gz 2023-07-06T01:22:20.567 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108157.log 2023-07-06T01:22:20.567 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108158.log 2023-07-06T01:22:20.568 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108157.log: /var/log/ceph/ceph-client.admin.108158.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108157.log.gz 2023-07-06T01:22:20.568 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108158.log.gz 2023-07-06T01:22:20.568 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111871.log 2023-07-06T01:22:20.568 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124277.log 2023-07-06T01:22:20.568 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111364.log 2023-07-06T01:22:20.569 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111871.log.gz 2023-07-06T01:22:20.569 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124277.log: /var/log/ceph/ceph-client.admin.111364.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124277.log.gz 2023-07-06T01:22:20.569 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123348.log 2023-07-06T01:22:20.569 INFO:teuthology.orchestra.run.smithi022.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.111364.log.gz 2023-07-06T01:22:20.569 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107152.log 2023-07-06T01:22:20.570 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100982.log 2023-07-06T01:22:20.570 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.100982.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100982.log.gz 2023-07-06T01:22:20.571 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-07-06T01:22:20.571 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-07-06T01:22:20.571 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-mds.b.log: 92.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-07-06T01:22:20.578 INFO:teuthology.orchestra.run.smithi103.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-07-06T01:22:20.579 INFO:teuthology.orchestra.run.smithi103.stderr: 95.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-07-06T01:22:20.587 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123597.log 2023-07-06T01:22:20.588 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123348.log.gz 2023-07-06T01:22:20.588 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108155.log 2023-07-06T01:22:20.588 INFO:teuthology.orchestra.run.smithi022.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.107152.log.gz 2023-07-06T01:22:20.588 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123597.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.123597.log.gz 2023-07-06T01:22:20.588 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108155.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108155.log.gz 2023-07-06T01:22:20.589 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109673.log 2023-07-06T01:22:20.589 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114056.log 2023-07-06T01:22:20.589 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.109673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114600.log 2023-07-06T01:22:20.589 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109673.log.gz 2023-07-06T01:22:20.589 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108967.log 2023-07-06T01:22:20.590 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114056.log.gz 2023-07-06T01:22:20.590 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115258.log 2023-07-06T01:22:20.590 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114600.log.gz 2023-07-06T01:22:20.590 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108967.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108967.log.gz 2023-07-06T01:22:20.590 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115258.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115258.log.gz 2023-07-06T01:22:20.591 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111410.log 2023-07-06T01:22:20.591 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114926.log 2023-07-06T01:22:20.591 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111410.log: /var/log/ceph/ceph-client.admin.114926.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111410.log.gz 2023-07-06T01:22:20.591 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114926.log.gz 2023-07-06T01:22:20.591 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-06T01:22:20.592 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108524.log 2023-07-06T01:22:20.592 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112101.log 2023-07-06T01:22:20.592 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108524.log: /var/log/ceph/ceph-client.admin.112101.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108524.log.gz 2023-07-06T01:22:20.592 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112101.log.gz 2023-07-06T01:22:20.592 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113404.log 2023-07-06T01:22:20.593 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-06T01:22:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114148.log 2023-07-06T01:22:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107767.log 2023-07-06T01:22:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114148.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113404.log.gz 2023-07-06T01:22:20.593 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107767.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114148.log.gz 2023-07-06T01:22:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122934.log 2023-07-06T01:22:20.594 INFO:teuthology.orchestra.run.smithi022.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107767.log.gz 2023-07-06T01:22:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107772.log 2023-07-06T01:22:20.594 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.122934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113132.log 2023-07-06T01:22:20.594 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122934.log.gz/var/log/ceph/ceph-client.admin.107772.log: 2023-07-06T01:22:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113132.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107772.log.gz 2023-07-06T01:22:20.595 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113132.log.gz 2023-07-06T01:22:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123394.log 2023-07-06T01:22:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115072.log 2023-07-06T01:22:20.595 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107290.log 2023-07-06T01:22:20.595 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123394.log.gz 2023-07-06T01:22:20.596 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115072.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115072.log.gz 2023-07-06T01:22:20.596 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112009.log 2023-07-06T01:22:20.596 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107290.log.gz 2023-07-06T01:22:20.596 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114508.log 2023-07-06T01:22:20.596 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.112009.log: /var/log/ceph/ceph-client.admin.114508.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112009.log.gz 2023-07-06T01:22:20.597 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114508.log.gz 2023-07-06T01:22:20.597 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106624.log 2023-07-06T01:22:20.597 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106678.log 2023-07-06T01:22:20.597 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.106624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112707.log 2023-07-06T01:22:20.597 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.106678.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106624.log.gz 2023-07-06T01:22:20.598 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.112707.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106678.log.gz 2023-07-06T01:22:20.598 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112707.log.gz 2023-07-06T01:22:20.598 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112055.log 2023-07-06T01:22:20.598 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114830.log 2023-07-06T01:22:20.598 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.112055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109427.log 2023-07-06T01:22:20.599 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114830.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112055.log.gz 2023-07-06T01:22:20.599 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.109427.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.114830.log.gz 2023-07-06T01:22:20.599 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109427.log.gz 2023-07-06T01:22:20.599 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111963.log 2023-07-06T01:22:20.599 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122735.log 2023-07-06T01:22:20.600 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100791.log 2023-07-06T01:22:20.600 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.122735.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111963.log.gz 2023-07-06T01:22:20.600 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.100791.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122735.log.gz 2023-07-06T01:22:20.600 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100791.log.gz 2023-07-06T01:22:20.600 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106826.log 2023-07-06T01:22:20.601 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112320.log 2023-07-06T01:22:20.601 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.106826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112753.log 2023-07-06T01:22:20.601 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.112320.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106826.log.gz 2023-07-06T01:22:20.601 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.112753.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112320.log.gz 2023-07-06T01:22:20.601 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112753.log.gz 2023-07-06T01:22:20.602 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124185.log 2023-07-06T01:22:20.602 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106315.log 2023-07-06T01:22:20.602 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115166.log 2023-07-06T01:22:20.602 INFO:teuthology.orchestra.run.smithi022.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124185.log.gz 2023-07-06T01:22:20.602 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.106315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124093.log 2023-07-06T01:22:20.602 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106315.log.gz 2023-07-06T01:22:20.603 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113542.log 2023-07-06T01:22:20.603 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115166.log.gz 2023-07-06T01:22:20.603 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124093.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124093.log.gz 2023-07-06T01:22:20.603 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113542.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113542.log.gz 2023-07-06T01:22:20.603 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111779.log 2023-07-06T01:22:20.604 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113450.log 2023-07-06T01:22:20.604 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107763.log 2023-07-06T01:22:20.604 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113450.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111779.log.gz 2023-07-06T01:22:20.604 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107763.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113450.log.gz 2023-07-06T01:22:20.605 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124859.log 2023-07-06T01:22:20.605 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107763.log.gz 2023-07-06T01:22:20.605 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108527.log 2023-07-06T01:22:20.605 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109243.log 2023-07-06T01:22:20.605 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124859.log.gz 2023-07-06T01:22:20.606 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108527.log: /var/log/ceph/ceph-client.admin.109243.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108527.log.gz 2023-07-06T01:22:20.606 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.109243.log.gz 2023-07-06T01:22:20.606 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123302.log 2023-07-06T01:22:20.607 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124599.log 2023-07-06T01:22:20.607 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115717.log 2023-07-06T01:22:20.607 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124599.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.123302.log.gz 2023-07-06T01:22:20.607 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115717.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124599.log.gz 2023-07-06T01:22:20.607 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115717.log.gz 2023-07-06T01:22:20.608 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.94504.log 2023-07-06T01:22:20.608 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107382.log 2023-07-06T01:22:20.608 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.tmp-client.admin.94504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108159.log 2023-07-06T01:22:20.608 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.94504.log.gz 2023-07-06T01:22:20.608 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122781.log 2023-07-06T01:22:20.609 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108159.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107382.log.gz 2023-07-06T01:22:20.609 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.122781.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108159.log.gz 2023-07-06T01:22:20.609 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122781.log.gz 2023-07-06T01:22:20.609 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113312.log 2023-07-06T01:22:20.609 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-07-06T01:22:20.610 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108162.log 2023-07-06T01:22:20.610 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113312.log.gz 2023-07-06T01:22:20.610 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113956.log 2023-07-06T01:22:20.610 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108162.log: 88.2%/var/log/ceph/ceph-client.admin.113956.log: -- replaced with /var/log/ceph/ceph-client.admin.108162.log.gz 2023-07-06T01:22:20.610 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113956.log.gz 2023-07-06T01:22:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109105.log 2023-07-06T01:22:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113772.log 2023-07-06T01:22:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.109105.log: /var/log/ceph/ceph-client.admin.113772.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109105.log.gz 2023-07-06T01:22:20.611 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113772.log.gz 2023-07-06T01:22:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115555.log 2023-07-06T01:22:20.611 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112594.log 2023-07-06T01:22:20.612 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115555.log: /var/log/ceph/ceph-client.admin.112594.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115555.log.gz 2023-07-06T01:22:20.612 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112594.log.gz 2023-07-06T01:22:20.612 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109473.log 2023-07-06T01:22:20.612 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122550.log 2023-07-06T01:22:20.612 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.109473.log: /var/log/ceph/ceph-client.admin.122550.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109473.log.gz 2023-07-06T01:22:20.613 INFO:teuthology.orchestra.run.smithi022.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122550.log.gz 2023-07-06T01:22:20.613 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106780.log 2023-07-06T01:22:20.613 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124691.log 2023-07-06T01:22:20.613 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.106780.log: /var/log/ceph/ceph-client.admin.124691.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106780.log.gz 2023-07-06T01:22:20.613 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124691.log.gz 2023-07-06T01:22:20.614 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114102.log 2023-07-06T01:22:20.614 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113496.log 2023-07-06T01:22:20.614 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114102.log: /var/log/ceph/ceph-client.admin.113496.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114102.log.gz 2023-07-06T01:22:20.614 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113496.log.gz 2023-07-06T01:22:20.614 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124323.log 2023-07-06T01:22:20.615 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123072.log 2023-07-06T01:22:20.615 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124323.log: /var/log/ceph/ceph-client.admin.123072.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124323.log.gz 2023-07-06T01:22:20.615 INFO:teuthology.orchestra.run.smithi022.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123072.log.gz 2023-07-06T01:22:20.615 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124461.log 2023-07-06T01:22:20.615 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.110155.log 2023-07-06T01:22:20.616 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124461.log: /var/log/ceph/ceph-client.0.110155.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124461.log.gz 2023-07-06T01:22:20.616 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.110155.log.gz 2023-07-06T01:22:20.616 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.110816.log 2023-07-06T01:22:20.616 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109197.log 2023-07-06T01:22:20.616 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.2.110816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.110816.log.gz 2023-07-06T01:22:20.617 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.109197.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.109197.log.gz 2023-07-06T01:22:20.617 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106964.log 2023-07-06T01:22:20.617 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113680.log 2023-07-06T01:22:20.617 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.106964.log: /var/log/ceph/ceph-client.admin.113680.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106964.log.gz 2023-07-06T01:22:20.617 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113680.log.gz 2023-07-06T01:22:20.618 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124737.log 2023-07-06T01:22:20.618 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124949.log 2023-07-06T01:22:20.618 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124737.log: /var/log/ceph/ceph-client.admin.124949.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.124737.log.gz 2023-07-06T01:22:20.618 INFO:teuthology.orchestra.run.smithi022.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.124949.log.gz 2023-07-06T01:22:20.618 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107520.log 2023-07-06T01:22:20.619 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124903.log 2023-07-06T01:22:20.619 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107520.log: /var/log/ceph/ceph-client.admin.124903.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107520.log.gz 2023-07-06T01:22:20.619 INFO:teuthology.orchestra.run.smithi022.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.124903.log.gz 2023-07-06T01:22:20.619 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111134.log 2023-07-06T01:22:20.619 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124369.log 2023-07-06T01:22:20.619 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111134.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111134.log.gz 2023-07-06T01:22:20.620 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.124369.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124369.log.gz 2023-07-06T01:22:20.620 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112274.log 2023-07-06T01:22:20.620 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113726.log 2023-07-06T01:22:20.620 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.112274.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112274.log.gz 2023-07-06T01:22:20.620 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.110820.log 2023-07-06T01:22:20.621 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113726.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113726.log.gz 2023-07-06T01:22:20.621 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.2.110820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.110820.log.gz 2023-07-06T01:22:20.621 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123863.log 2023-07-06T01:22:20.621 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112661.log 2023-07-06T01:22:20.621 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123863.log: /var/log/ceph/ceph-client.admin.112661.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123863.log.gz 2023-07-06T01:22:20.622 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112661.log.gz 2023-07-06T01:22:20.622 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123955.log 2023-07-06T01:22:20.622 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107762.log 2023-07-06T01:22:20.622 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123955.log: /var/log/ceph/ceph-client.admin.107762.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123955.log.gz 2023-07-06T01:22:20.622 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107762.log.gz 2023-07-06T01:22:20.623 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113818.log 2023-07-06T01:22:20.623 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114980.log 2023-07-06T01:22:20.623 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113818.log: 88.7%/var/log/ceph/ceph-client.admin.114980.log: -- replaced with /var/log/ceph/ceph-client.admin.113818.log.gz 2023-07-06T01:22:20.623 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114980.log.gz 2023-07-06T01:22:20.623 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113040.log 2023-07-06T01:22:20.624 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123458.log 2023-07-06T01:22:20.624 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113040.log: /var/log/ceph/ceph-client.admin.123458.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113040.log.gz 2023-07-06T01:22:20.624 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123458.log.gz 2023-07-06T01:22:20.624 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113588.log 2023-07-06T01:22:20.624 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108529.log 2023-07-06T01:22:20.625 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113588.log: /var/log/ceph/ceph-client.admin.108529.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113588.log.gz 2023-07-06T01:22:20.625 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108529.log.gz 2023-07-06T01:22:20.625 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115442.log 2023-07-06T01:22:20.628 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113910.log 2023-07-06T01:22:20.629 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115442.log: /var/log/ceph/ceph-client.admin.113910.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115442.log.gz 2023-07-06T01:22:20.629 INFO:teuthology.orchestra.run.smithi022.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113910.log.gz 2023-07-06T01:22:20.629 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108528.log 2023-07-06T01:22:20.629 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112948.log 2023-07-06T01:22:20.630 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108528.log: /var/log/ceph/ceph-client.admin.112948.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108528.log.gz 2023-07-06T01:22:20.630 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.112948.log.gz 2023-07-06T01:22:20.630 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114646.log 2023-07-06T01:22:20.630 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106918.log 2023-07-06T01:22:20.630 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114646.log: /var/log/ceph/ceph-client.admin.106918.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114646.log.gz 2023-07-06T01:22:20.631 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106918.log.gz 2023-07-06T01:22:20.631 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115120.log 2023-07-06T01:22:20.631 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122417.log 2023-07-06T01:22:20.631 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115120.log: /var/log/ceph/ceph-client.admin.122417.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.115120.log.gz 2023-07-06T01:22:20.631 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115026.log 2023-07-06T01:22:20.632 INFO:teuthology.orchestra.run.smithi022.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.122417.log.gz 2023-07-06T01:22:20.632 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115026.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115026.log.gz 2023-07-06T01:22:20.632 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111272.log 2023-07-06T01:22:20.632 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111825.log 2023-07-06T01:22:20.632 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111272.log: /var/log/ceph/ceph-client.admin.111825.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111272.log.gz 2023-07-06T01:22:20.633 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111825.log.gz 2023-07-06T01:22:20.633 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123026.log 2023-07-06T01:22:20.633 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114370.log 2023-07-06T01:22:20.633 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.123026.log: /var/log/ceph/ceph-client.admin.114370.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.123026.log.gz 2023-07-06T01:22:20.633 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114370.log.gz 2023-07-06T01:22:20.634 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109335.log 2023-07-06T01:22:20.634 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114238.log 2023-07-06T01:22:20.634 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.109335.log: /var/log/ceph/ceph-client.admin.114238.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109335.log.gz 2023-07-06T01:22:20.634 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114238.log.gz 2023-07-06T01:22:20.634 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-07-06T01:22:20.635 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111641.log 2023-07-06T01:22:20.635 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.111641.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111641.log.gz 2023-07-06T01:22:20.635 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114738.log 2023-07-06T01:22:20.635 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.114738.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114738.log.gz 2023-07-06T01:22:20.635 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108829.log 2023-07-06T01:22:20.636 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108829.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108829.log.gz 2023-07-06T01:22:20.636 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109519.log 2023-07-06T01:22:20.636 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.109519.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.109519.log.gz 2023-07-06T01:22:20.636 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115350.log 2023-07-06T01:22:20.636 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.115350.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115350.log.gz 2023-07-06T01:22:20.637 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107566.log 2023-07-06T01:22:20.637 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.107566.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107566.log.gz 2023-07-06T01:22:20.637 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113358.log 2023-07-06T01:22:20.637 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.113358.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113358.log.gz 2023-07-06T01:22:20.638 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-07-06T01:22:20.638 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mgr.y.log: 93.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-07-06T01:22:20.638 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.115954.log 2023-07-06T01:22:20.638 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.115954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.115954.log.gz 2023-07-06T01:22:20.638 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108154.log 2023-07-06T01:22:20.639 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.108154.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108154.log.gz 2023-07-06T01:22:20.639 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111503.log 2023-07-06T01:22:20.639 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.111503.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111503.log.gz 2023-07-06T01:22:20.639 INFO:teuthology.orchestra.run.smithi103.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-07-06T01:22:20.670 INFO:teuthology.orchestra.run.smithi103.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-07-06T01:22:20.799 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-07-06T01:22:21.164 INFO:teuthology.orchestra.run.smithi103.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-07-06T01:22:21.188 INFO:teuthology.orchestra.run.smithi022.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-07-06T01:22:21.224 INFO:teuthology.orchestra.run.smithi103.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-07-06T01:22:21.320 INFO:teuthology.orchestra.run.smithi103.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-07-06T01:22:21.340 INFO:teuthology.orchestra.run.smithi103.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-07-06T01:22:21.342 INFO:teuthology.orchestra.run.smithi103.stderr: 2023-07-06T01:22:21.342 INFO:teuthology.orchestra.run.smithi103.stderr:real 0m0.839s 2023-07-06T01:22:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:user 0m3.227s 2023-07-06T01:22:21.343 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m0.132s 2023-07-06T01:22:21.350 INFO:teuthology.orchestra.run.smithi022.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-07-06T01:22:21.356 INFO:teuthology.orchestra.run.smithi022.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-07-06T01:22:21.441 INFO:teuthology.orchestra.run.smithi022.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-07-06T01:22:44.772 INFO:teuthology.orchestra.run.smithi022.stderr: 94.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-07-06T01:22:44.773 INFO:teuthology.orchestra.run.smithi022.stderr: 2023-07-06T01:22:44.774 INFO:teuthology.orchestra.run.smithi022.stderr:real 0m24.298s 2023-07-06T01:22:44.774 INFO:teuthology.orchestra.run.smithi022.stderr:user 0m27.025s 2023-07-06T01:22:44.774 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m0.913s 2023-07-06T01:22:44.775 INFO:tasks.ceph:Archiving logs... 2023-07-06T01:22:44.775 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789/remote/smithi022/log 2023-07-06T01:22:44.776 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-06T01:22:47.848 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789/remote/smithi103/log 2023-07-06T01:22:47.849 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-06T01:22:48.265 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-06T01:22:48.281 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-06T01:22:48.281 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-06T01:22:48.285 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-06T01:22:48.377 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-06T01:22:48.390 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-06T01:22:48.390 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T01:22:48.393 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T01:22:48.410 INFO:teuthology.orchestra.run.smithi103.stderr:bash: ntpq: command not found 2023-07-06T01:22:48.411 INFO:teuthology.orchestra.run.smithi022.stderr:bash: ntpq: command not found 2023-07-06T01:22:48.414 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T01:22:48.414 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2023-07-06T01:22:48.415 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 2 -564us[ -564us] +/- 102ms 2023-07-06T01:22:48.415 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv02.front.sepia.ceph.com 3 6 377 2 -446us[ -441us] +/- 64ms 2023-07-06T01:22:48.415 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +1025us[+1030us] +/- 79ms 2023-07-06T01:22:48.415 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T01:22:48.416 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T01:22:48.417 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2023-07-06T01:22:48.417 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 63 -567us[ -572us] +/- 101ms 2023-07-06T01:22:48.417 INFO:teuthology.orchestra.run.smithi022.stdout:^* hv02.front.sepia.ceph.com 3 6 377 2 -475us[ -482us] +/- 64ms 2023-07-06T01:22:48.417 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 0 +1049us[+1049us] +/- 79ms 2023-07-06T01:22:48.418 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T01:22:48.418 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-06T01:22:48.430 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-06T01:22:48.431 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-06T01:22:48.440 DEBUG:teuthology.orchestra.run.smithi022:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-06T01:22:48.564 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-06T01:22:48.657 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T01:22:48.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:22:48.692 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T01:22:48.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:22:48.724 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-06T01:22:48.740 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-06T01:22:48.750 INFO:teuthology.task.internal:Duration was 1021.078507 seconds 2023-07-06T01:22:48.751 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-06T01:22:48.765 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-06T01:22:48.765 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-06T01:22:48.768 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-06T01:22:48.815 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T01:22:48.816 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T01:22:49.159 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-06T01:22:49.160 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2023-07-06T01:22:49.160 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-06T01:22:49.184 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2023-07-06T01:22:49.184 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-06T01:22:49.206 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-06T01:22:49.207 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-06T01:22:49.225 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-06T01:22:49.287 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-06T01:22:49.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-06T01:22:49.310 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-06T01:22:49.401 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-06T01:22:49.414 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-06T01:22:49.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-06T01:22:49.443 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-06T01:22:49.470 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-06T01:22:49.481 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-06T01:22:49.486 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-06T01:22:49.510 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2023-07-06T01:22:49.536 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2023-07-06T01:22:49.549 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-06T01:22:49.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:22:49.581 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-06T01:22:49.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T01:22:49.606 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-06T01:22:49.622 INFO:teuthology.task.internal:Transferring archived files... 2023-07-06T01:22:49.623 DEBUG:teuthology.misc:Transferring archived files from smithi022:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789/remote/smithi022 2023-07-06T01:22:49.623 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-06T01:22:49.662 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789/remote/smithi103 2023-07-06T01:22:49.662 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-06T01:22:49.697 INFO:teuthology.task.internal:Removing archive directory... 2023-07-06T01:22:49.698 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-06T01:22:49.700 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-06T01:22:49.749 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-06T01:22:49.763 INFO:teuthology.task.internal:Not uploading archives. 2023-07-06T01:22:49.764 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-06T01:22:49.783 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-06T01:22:49.784 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-06T01:22:49.786 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-06T01:22:49.801 INFO:teuthology.orchestra.run.smithi022.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 6 01:22 /home/ubuntu/cephtest 2023-07-06T01:22:49.805 INFO:teuthology.orchestra.run.smithi103.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 6 01:22 /home/ubuntu/cephtest 2023-07-06T01:22:49.807 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-06T01:22:49.818 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-06T01:22:49.838 INFO:teuthology.nuke:Checking targets against current locks 2023-07-06T01:22:49.869 DEBUG:teuthology.nuke:shortname: smithi022 2023-07-06T01:22:49.869 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T01:22:49.895 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 01:00:04.018713', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-06T01:22:49.901 DEBUG:teuthology.nuke:shortname: smithi103 2023-07-06T01:22:49.902 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T01:22:49.927 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327789', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 01:00:04.020694', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-06T01:22:49.946 INFO:teuthology.orchestra.console:Power off smithi022 2023-07-06T01:22:49.947 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-06T01:22:49.971 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T01:22:49.995 INFO:teuthology.orchestra.console:Power off smithi103 2023-07-06T01:22:49.996 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-06T01:22:50.020 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T01:22:54.009 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T01:22:54.050 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T01:22:58.124 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T01:22:58.163 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T01:22:58.337 INFO:teuthology.orchestra.console:Power off for smithi022 completed 2023-07-06T01:22:58.376 INFO:teuthology.orchestra.console:Power off for smithi103 completed 2023-07-06T01:22:58.481 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} duration: 1021.0785074234009 failure_reason: 'Test failure: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=53471a5faa414ff7a03a682b498ee615 status: fail success: false 2023-07-06T01:22:58.482 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-06T01:22:58.570 INFO:teuthology.run:FAIL